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PREFACE 


This is the Experimental and Software Detailed Design Report for the prototype Task Loading 
Model (TLM) developed as part of the Man-Machine Integration Design and Analysis System, as 
implemented and tested in Phase VI of the Army-NASA Aircrew/ Aircraft Integration (A3I) 
Program. The A3I Program is a joint Army and NASA exploratory development effort to 
advance the capabilities and use of computational representations of human performance and 
behavior in the design, synthesis, and analysis of manned systems. The MIDAS TLM 
computationally models the demands designs impose on operators to aid engineers in the 
conceptual design of aircraft crewstations. This report describes TLM and the results of a series 
of experiments which were run this phase to test its capabilities as a predictive task demand 
modeling tool. Specifically, it includes discussions of: the inputs and outputs of the TLM; the 
theories underlying it; the results of the test experiments; the use of the TLM as both standalone 
tool and part of a complete human operator simulation; and a brief introduction to the TLM 
software design. 
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1.0 INTRODUCTION 


Task Load Prediction is a critical aspect of design; It is a construct related to performance measures 
and subjective workload ratings that may help spot design defects. Figure 1 depicts how the 
construct of task load relates to workload and performance from the MIDAS modeling perspective. 



Figure 1-1: An overview of MIDAS Task Load Modeling 

The MIDAS Task Loading Model computationally models the demands designs impose on 
operators to aid engineers in the conceptual design of aircraft crewstations. 

• Load = Task Load = % resources required to meet demands. 

• Loads modeled on four dimensions: Visual, Auditory, Cognitive, Motor. 


1.1 Identification of Report 

This reports details the TLM Experimental results conducted to test the TLM as a predictive task 
demand modeling tool. 

1.2 Scope of Report 

This report describes all of the aspects of the TLM. It describes the following in detail: 


The inputs and outputs of the TLM. 










• The theories underlying the TLM. 

• The experimental results from testing the TLM. 

• Its use as a standalone and simulation modeling tool. 

• A brief introduction to the TLM software design. 

The readers of this report should have knowledge of human performance modeling techniques, 
human information processing theories, and some experience designing and analyzing 
experiments. 

Refer to the MIDAS Software detailed design document for a complete report on the 
implementation of the TLM (Banda, C., Bushnell, D., Chen, S., Chiu, A., Neukom, C., 
Nishimura, S., Pisanich, G., Prevost, M., Shankar, R., Staveland, L., & Smith, G., 1991). 

1.3 Objectives of Report 

Readers will learn the following: 

• How the TLM classifies task attributes. 

• How the TLM represents imposed task demands and conflicts. 

• How the TLM calculates the task load estimates. 

• How well TLM task load estimates correlate with empirical performance and workload data. 
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2.0 


TLM OVERVIEW 


2 . 1 Goal of TLM 

The overall goal of the MIDAS-TLM is to predict the information processing demands on each of 
four psychological dimensions that activities associated with a conceptual system design impose on 
the system’s operator. The demands are used to calculate an estimate of task loading, allowing the 
system’s designers to evaluate how a conceptual design affects information processing demands. 
The four dimensions are Visual, Auditory, Cognitive, Motor, abbreviated as VACM throughout 
this report. 

Tasks and activities are used interchangeably in this report. They represent a low level procedure, 
operator movement, decision process or task that satisfies a goal. 

2.2 Scope of TLM 

The MIDAS-TLM expands the previously used MIDAS loading model to include an evaluation of 
the loads that are placed on pilots interacting with a given design within the context of a series of 
flight activities or tasks that are generated during a simulated flight. The TLM is an extension of 
computational workload modeling and simulation efforts by Wickens (1984), McCracken & 
Aldrich (1984), North & Riley (1989). 

2.2.1 TLM as a Simulation Tool 

The TLM computationally links with other MIDAS workstation tools and models that render a 
conceptual crewstation design and analyze the design for the activities required to operate it. The 
analyst classifies the activities using the TLM’s task attributes and the MIDAS simulation changes 
the attributes to match the run time context. Estimated loads are calculated from the run time 
classifications. 

2.2.2 TLM as a Standalone Tool 

The TLM can be combined with a CAD tool and mission and task analysis tool to render a 
conceptual crewstation design and analyze the design for the activities required to operate it. The 
analyst classifies the activities using the TLM’s task attributes. Estimated loads are calculated from 
the analyst classifications. 

2.3 Definition of Task Loading 

Task loading is defined as the aircrew's or operators' capabilities to perceive and process the 
information imposed on their perceptual, cognitive and motor systems by activity demands. This 
definition is based on the assertion of information processing which holds that human performance 
can be objectively and quantitatively described with information processing structures in 
conjunction with the mental processes that act on those structures (Wickens & Flach, 1988; 
Lachman, Lachman & Butterfield, 1979; Kantowitz & Roediger, 1980; Posner, 1986: Chase, 
1986). This assertion leads to the following assumptions that form the basis of the TLM 
taxonomy, and the demand and conflict structures. 

1 . A structure is a symbolic representation of information. 

2 . A process is a manipulation of that representation. 

3 . A limited set of structures and processes represents the necessary task attributes for 
sufficiently estimating loads. 
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4 . The structures and processes require resources to function. 

5 . The resources are attention, memory, and time. 

6 . Demands represent the percentage of the resources required for task performance. 

7 Conflicts represent the imposition of similar demands required for concurrent task 

performance. 

The demands are rank-ordered according to resource use. The task load estimates are calculated 
from the demands. 

These demand values represent the relative magnitudes of the demands imposed on the person 
performing the activity(s). The magnitudes are anchored to a 100 point scale to differentiate load 
values among activities relative to other tasks, not to an absolute scale. Therefore, high load values 
represent potentially high demands. Potentially, because the demands are relative among tasks. If 
the average demands are low, then peaks may represent more demands, but may not represent high 
demands. 

2.4 Definition of TLM Model 

The model is an output, normative, bottom-up, multi-task model of imposed task demands. It is 
an output model because it generates the loading values after being fed a task description. It is 
normative because it assumes that the aircrews or system operators are highly skilled and 
motivated, and would perform in a manner that is rational and consistent with the information 
available, and with the constraints, risks and objectives that exist. It is bottom-up because it 
generates the values based on rank orderings of the interactions and combinations of basic 
perceptual, cognitive, and motor activities. In this sense, it also has some process and prescription 
characteristics, because the basic activities can be diagnostic of the problems in the conceptual 
designs (indicated by high loading values). It is a multi-task model because it evaluates the loading 
of a variety of tasks performed concurrently as well as serially. 

2.5 Structure of TLM 

Figure 2-1 graphically depicts the structural overview of the TLM. The top half of the graphic 
depicts the models functional structure, the bottom half depicts the computational structure. Both 
the functional structure and computational structure are depicted in three dimensions. The x-axis 
depicts processing from input to output. The y-axis depicts model attributes, either functional or 
computational. The z-axis depicts the tasks relevant to the analysis. 

In the top half of Figure 2-1, the model’s functional structure, the x-axis depicts the stages of 
information processing from the start to finish of a task, in which the stimuli are mapped to a 
response. The three stages are input modality (auditory and visual), central processing (response 
selection), and output modality (either verbal or manual response execution). The y-axis depicts 
both the demand values and the conflict values assigned to each dimension for each of the three 
stages. Visual and Auditory are in the input modality stage, .Cognitive is in the response selection 
stage, and Motor is in the response execution stage. The demand values are partially listed. The 
conflict values (also partially listed) are depicted for the attributes of each task (shown in the z-axis) 
associated with each stage: reading multiple gauges are shown as separate tasks for the visual 
dimension. Choosing between the best responses given the different gauge readings is shown in 
the processing stage. Executing the correct movement is shown in the response execution stage. 

The bottom half of Figure 2- 1 shows how the TLM computationally represents the top half. The x- 
axis depicts in three steps how the TLM estimates loads from the analysts classification of the 
stimuli (input) to the estimates of demands and conflicts (processing) to the estimated load values 
for each dimension (output). The analyst classifies the stimuli of the tasks depicted in the Z-axis 
using the attribute taxonomy partially depicted in the y-axis. Table 2-1 lists the full taxonomy. This 
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Figure 2-1. The structural design of the TLM. 


2.6 Generation of Task Load Values 

Generating load estimates requires three steps: classifying a task, listing the demand and conflict 
values referenced by the classification, and then calculating the load estimates from the lists of 
demand and conflict values. In step 1, an analyst classifies tasks, using a taxonomy of perceptual, 
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cognitive and motor attributes (Table 2-1). The taxonomy parses imposed task demands into four 
dimensions; each dimension is parsed into sets of binary task attributes (only one member of a pair 
per task). Based on a literature review, the memory, attention, and time demands and conflicts 
between attributes in this taxonomy were rank ordered. Note that in the TLM the rankings do not 
change across tasks or task contexts. 

Table 2-1: Taxonomy of Visual, Auditory, Cognitive and Motor Classifications. 


Visqal 

Auditory 

near visual 

orient 

far visual 

discriminate 

scan 

signal 

fixate 

speech 

signal-ratio 

signal-ratio 

noise-ratio 

noise-ratio 

salient 

temporal-location 

masked 

physical-location 

static 


dynamic 



Cognitive 

Motpr 

direct 

verbal 

transformation 

spatial 

single choice 

near 

multiple choice 

far 

verbal 

discrete 

spatial 

continuous 

separable 

gross 

integral 

fine 

planned 


unplanned 



In step 2, the TLM creates lists of demand and conflict values from matrices of demand and 
conflict values using the classifications as matrix accessors. In step 3, the TLM algorithms are 
applied to the lists to generate load estimates for each VACM dimension. 

2.7 Strengths of the TLM 

First, and most importantly, the individual task demand values do not need to be specified — they 
are derived by the model. This frees a designer from having to specify the task demand values for 
each new set of tasks required to simulate a design. However, these values can be tailored to a 
testing environment by re-coding demand and conflict matrices before being used. 

Second, by classifying a task, a designer/analyst makes explicit the information required and the 
kind of processing involved in performing a flight task. 

Third, the conflict matrices explicitly predict conflicts between the demands required to perform the 
flight task.. 

Fourth, the classifications are context sensitive because they are adjusted to match the run time 
context. The classification of one task can influence the classification of another task. 

Fifth, this approach allow for the loads a design imposes on an operator to be estimated during the 
conceptual design of the aircraft cockpit (or other complex system). These predictions normally 
occur late in the design phase after the requisite system mockups are built. 
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2.8 Limitations Of the TLM 


1) In general, the TLM and MIDAS are dependent on the quantity and quality of the simulation 
state information encoded by analysts using MIDAS to build and simulate design scenarios. As a 
result, the simulation state information that determines imposed loads represents a subset of the 
possible state information that changes as the simulation progresses. Currently, this information 
adjusts only the Visual and Cognitive dimensions. 

There were two reasons for this: One, MIDAS lacks a model of audition that can generate 
information useful to the TLM, and two, MIDAS lacks an operator continuous manual control 
model that can generate the appropriate psycho-motor information. 

2) All VACM load values are relative. Do not treat them as absolutes. Values exceeding 100 mean 
little unless compared to all the activities in the overall load trace, and should be compared to 
additional simulation state information to determine the load drivers. 

3) The weakness in this modeling approach lies in its testing requirements. This model uses 
complex classifications and algorithms to generate loading values that are difficult and time 
consuming to validate. 

The values have only been tested four times: once in an experiment, and three times by retrofitting 
TLM predictions to empirical data: the first data were from an experiment conducted by Samo and 
Wickens (1991), the second and third sets of data were from experiments conducted by Andre 
(1993). A fifth test currently underway uses an instrument flight simulator 

2.9 Future Directions 

A final experiment, described in section 7.0 is currently being conducted. However, no further 
extensions to the TLM are planned as part of the MIDAS program. 
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3.0 DETAILS OF VACM TAXONOMY 

3.1 Generating Classifications 

The taxonomy in Table 2-1 lists the attributes used to classify activities; as the table shows it is 
composed of four dimensions, each consisting a set of paired attributes. One attribute from each 
pair of attributes from each dimension can be used to classify an activity. However, one attribute 
from every pair does not have to be selected. Only those attributes that are relevant to the activity 
being classified are required. The definitions of the attributes used to determine relevancy are listed 
in section 3.3 (brief definitions) and 3.4 (detailed definitions). 

The attributes selected from each dimension constitute the classification for an activity. The selected 
attributes are assigned a unique number within a dimension (numbers are re-used between 
dimensions). Within a dimension the numbers increase from 1 to n, where 1 is assigned to the first 
member of the first attribute pair, and n equals the number assigned to the second member of the 
last pair. Table 2, next page, lists the numbers assigned to the attributes. 

The numbers assigned to each activity are represented as a nested list - a list of four sub-lists 

For example, ((1 3 5 7 9)(2 4)(1 4 5 8 10)(2 4 6))). 

The list of numbers assigned to an activity are passed as arguments to the TLM algorithms, the 
numbers index the task demand values and the conflict values that feed into algorithms that 
calculate the load values on each dimension for every activity or set of activities. 

3.2 The Taxonomy Spreadsheet 

Table 3-1 shows a taxonomic spreadsheet that can be used to manually classify each task or task 
combination. Task names are written in the spaces at the top of each column. The names of each 
attribute are listed in the left column, one attribute per row. To classify a task, the analyst uses 
available knowledge of task procedures and equipment to select one attribute from each pair of 
attributes listed. The number to the right of the attribute name is entered into the cells along that 
attribute’s row, only in the columns that correspond to the tasks being classified. All of the 
numbers in a column become the classification list for the task listed at the top of the column. The 
lists of numbers become the row and column accessors to the matrices of demands and conflicts. 
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Table 3-1: The VACM full taxonomic classification spreadsheet. 


TASKS 


ELEMENTS 



siqnal - ratio 5 
noise - ratio 6 


salient 7 

masked 8 


static 9 

dynamic 1C 



sinqle choice 3 
multiple choice 4 

verbal 5 

atial 6 


separable 7 

integral 8 



discrete 

continuous 


gross 

fine 
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3.3 Brief Definitions Of The Taxonomic Attributes. 


VISUAL-DIMENSION 

NEAR: 

Near visual field classification. 

FAR: 

Far visual field classification. 

SCAN: 

Fixating multiple stimuli. 

FIXATE: 

Fixating single stimuli. 

SIGNAL-RATIO: 

Greater stimulus signal. 

NOISE-RATIO: 

Greater noise signal. 

SALIENT: 

Strong stimulus signal. 

MASKED: 

Weak stimulus signal. 

STATIC: 

Stationary stimulus. 

DYNAMIC: 

Moving stimulus. 

AUDITORY-DIMENSION 

ORIENT: 

Locate stimulus. 

DISCRIMINATE: 

Identify stimulus. 

SIGNAL: 

Non-verbal stimulus. 

SPEECH: 

Verbal stimulus. 

SIGNAL-RATIO: 

Greater stimulus signal. 

NOISE-RATIO: 

Greater noise signal. 

TEMPORAL-LOCATION: 

Localize stimulus in time. 

PHYSICAL-LOCATION: 

Localize stimulus in space. 

COGNITIVE-DIMENSION 

DIRECT: 

Unprocessed stimulus mapped directly to a response. 

TRANSFORM: 

Stimulus needs processing to map to response. 

SINGLE-CHOICE: 

Stimulus requires processing. 

MULTIPLE-CHOICE: 

Stimulus requires complex processing. 

VERBAL: 

Stimulus is verbally encoded. 

SPATIAL: 

Stimulus is spatially encoded. 

SEPARABLE: 

Stimulus is uni-dimensional. 

INTEGRAL: 

Stimulus is multi-dimensional. 

PLANNED: 

Stimulus requires processing that maps to current situation. 

UNPLANNED: 

Stimulus requires processing that maps to different situation. 

MOTOR-DIMENSION 

VERBAL: 

Verbal response required. 

SPATIAL: 

Spatial response required. 

NEAR: 

Input device close to effector. 

FAR: 

Input device not close to effector. 

DISCRETE: 

Short duration responses. 

CONTINUOUS: 

Long duration responses. 

GROSS: 

Large tolerances or motion in motor control. 

FINE: 

Narrow tolerances or small range of motion in motor control. 


3.4 Detailed Definitions Of The Taxonomic Attributes. 


Visual: Each visual taxonomic classifier (ie., scan, fixate, near, far) is a visual, perceptual 
attribute, usually referred to as a visual attribute, or just attribute. The attributes in the visual 
dimension characterize the information that is present in the optic array that is pertinent to the 
activity. This information is referred to as the visual stimuli, and the attributes classify the different 
aspects of the optic array. 

Near versus Far : Defining near-visual as the visual stimuli in the cockpit and far-visual as the 
visual stimuli outside of the cockpit divides the spatial location of the stimuli in the optic array into 
two course-distinctions. The stimuli in the cockpit can be divided into quadrants (left, right, front, 
and back) to make finer distinctions between the locations. Transitions within a quadrant are 
classified as near, and transitions between quadrants are classified as far. The stimuli outside the 
cockpit can be divided in a similar fashion with classifications dependent on viewing distance 
instead of cockpit quadrant. 

Scan versus Fixate : Classifying an activity as scan or fixate depends on the information acquisition 
process on the part of the pilot. In other words, the optic array may be sampled once or multiple 
times to acquire the stimulus or stimuli (a piece of equipment or object in the terrain). If multiple 
sampling occurs, then scan applies. Multiple sampling can occur within a display as well as across 
a display. If single sampling, fixate applies. Scan and Fixate are dependent on the level of activity 
decomposition (the extent of the detail of the activity definition). A multiple sampling can be a 
fixate if the information source, such as a HUD, is treated as a uni-dimensional source (a single 
stimuli), as in perceive HUD. However, if the activity is to interpret HUD, then scan applies 
because the pilot may have to individually sample each symbol on the HUD, treating each as a 
separate source of information (separate stimuli) that has to be sequentially fixated on to determine 
the status of the aircraft. The level of detail depends on the analytical questions asked. 

Signal-to-Noise Ratio : These attributes classify only the energetic strength of the stimulus; they do 
not classify the operators knowledge of the stimulus, which helps set the operators thresholds for 
perceiving the stimulus. Accordingly, these attributes are not traditional measures of intelligibility 
or signal detection; however, these attributes capture more in the traditional sense when they are 
used to classify activities in conjunction with attributes in the cognitive dimension. Basically, the 
Signal-to-Noise Ratio attributes account for the imposed demands relating to whether the stimulus 
is discriminable: if it is, what the chance is of identifying it and if it is not, what the chance is of 
detecting it. The more discriminable the stimulus, the greater the signal to noise ratio, and 
conversely, the less discriminable the stimulus, the greater the noise to signal ratio. 

Salient versus Masked : These attributes classify the objective, physical discriminibility of the 
stimuli, not the cognitive discriminibility. Cognitive salience is assumed if the object is physically 
discriminable (results of the simulation must be analyzed for the presence or absence of cognitive 
salience). Salience depends on the discriminibility of the stimuli with regard to the time-of-day, 
contrast, legibility, font size, display clutter, glare, spatial position (hard-to-see position in the 
cockpit), occlusion, weather conditions, movement, strength of perceptual grouping (possibly 
measured by the stimuli's coherence to gestalt principles), and other stimuli. If it's a night 
scenario, the displays may be salient due to cockpit lighting or masked due to incompatibility with 
night vision systems, and the external (far) visual stimuli may be masked due to darkness or salient 
if viewed through night vision systems. Conversely, in a daytime scenario external stimuli may be 
salient, or masked if there is fog. Internal (near/cockpit) stimuli may be masked due to glare, or 
they may be salient if glare is not a factor or if glare shields are used. If a display uses the proper 
font size, and with adequate contrast in daylight conditions providing legible display symbols, the 
visual stimuli would probably be salient. This classification depends on the simulated internal and 
external environmental conditions. 



Static versus Dynamic : Static and Dynamic are fairly straight-forward attributes. A visual stimulus 
is static if it is stationary, if there is movement involved it is dynamic. It only becomes complicated 
if, for example, the compass rose of a horizontal situation display is moving, but the whole display 
is viewed as a static unit, and the individual display attributes (such as the moving compass rose) 
aren’t the visual stimuli of interest. 

Auditory: The auditory attributes are treated similarly to the visual attributes. Each auditory 
taxonomic classifier (ie., orient, discriminate) is an auditory, perceptual attribute, referred to as an 
auditory attribute, or just attribute. The attributes in the auditory dimension characterize the 
information that is present in the aural array that is pertinent to the activity. This information will be 
referred to as the auditory stimuli, and these attributes classify the different aspects of the aural 
array. 

Orient versus Discriminate: Orient and Discriminate are attributes that classify the extent to which a 
pilot must process the aural stimuli. Orient refers to simply detecting or being alerted to the 
direction of a sound and or the general nature of the stimuli. Discriminate refers to identifying the 
stimuli by pinpointing the location in space, recognizing or recalling the specific kind of signal, and 
discriminating between different signals based on the direction, kind (e.g., frequency, amplitude, 
pitch), or temporal properties (simultaneous occurrence to cause grouping, or sequential 
occurrence to inhibit grouping and cause serial processing). 

Signal versus Speech : Signal and Speech classify the type of aural stimulus. Signal is any kind of 
non-linguistic (non-verbal) stimulus. Speech is any kind of linguistic stimulus (machine generated, 
recorded or spoken). 

Sipnal-to-Noise Ratio : These attributes classify only the energetic strength of the stimulus; they do 
not classify the operators knowledge of the stimulus, which help set the operators thresholds for 
perceiving the stimulus. Accordingly these attributes are not traditional measures of intelligibility or 
signal detection; however, these attributes capture more in the traditional sense when they are used 
to classify activities in conjunction with attributes in the cognitive dimension. Basically, the 
attributes account for the imposed demands relating to whether the stimulus is discriminable: if it 
is, what the chance is of identifying it and if it is not, what the chance is of detecting it. The more 
discriminable the stimulus, the greater the signal to noise ratio; conversely, the less discriminable 
the stimulus, the greater the noise to signal ratio. 

Temporal Location versus Physical Location : These attributes classify the proximity and 
distribution of the stimulus in time and space. Temporal location captures whether the auditory 
stimulus is temporally proximal or temporally distant. Physical location captures whether the 
stimulus is physically proximal or distant from either ear. It also captures its direction. 

Cognitive: The cognitive attributes classify the kind and extent of mental processing that is 
brought to bear on the stimuli. These attributes represent processes that act on the stimuli that have 
been filtered through the visual and auditory input modalities. These processes map the stimulus to 
the appropriate response(s). Activities should be subjected to a cognitive task analysis or a similar 
analytic activity in order to determine the activity details sufficient to assign a cognitive 
classification. Conversely, the definitions of the cognitive attributes also can be used to guide the 
analyst. 

Direct versus Transformation : These attributes classify the stimulus-response mapping 
requirements. If the stimulus automatically evokes a response, a direct classification applies. If the 
stimulus needs to be modified or changed, a transformation classification applies. Automatic 
mappings may be activities, such as flight control under nominal conditions, in which changes in 
the aircraft's attitude are rapidly compensated, without interfering with another activity. If the flight 
control activity occurs in a severe thunderstorm, it could be classified as transformation since the 



stimulus (visual or haptic information) may at that time require extensive attention (an indicator of 
processing) to efficiently compensate for changes in attitude or changes in latitude. 

Transformations can be rotating visual input, recall or recognition of information to identify the 
stimulus, re-coding a visual image into an appropriate mental image (transforming infrared to 
normal "television view"), pattern recognition, computations, triangulations and other 
transformations. The stimulus may be hard to classify using direct and transformation definitions 
without the analyst specifying what information is required and how it will be used to perform the 
activities (i.e. a cognitive task analysis). 

Single Choice versus Multiple Choice : These attributes roughly classify the amount of processing 
required. Essentially, Single Choice refers to one transformation or decision. It is the "easy level, 
or least amount of processing required. If the activity is classified as a direct activity (see previous 
paragraph - direct versus transformation), then Single Choice is not used in the classification 
because direct assumes no choice - it is an automatic stimulus-response mapping process. If more 
than one transformation of the stimulus, or a lot of memory access, or more than one decision is 
required, the classification is Multiple Choice. 

Verbal versus Spatial : These two attributes classify the form of the mental code in which the 
stimulus is received or processed. If the stimulus is linguistic, a verbal classification applies (digital 
information, alpha-numeric characters, symbols with linguistic or syntactic meaning). If the 
stimulus is not verbal, it’s spatial - probably the most straightforward way to define it, since verbal 
is the more constrained and manageable set. Spatial stimuli can be symbols referring to spatial 
locations, symbols that indicate appropriate responses (pitch ladders), and analogue indicators of 
state information (airspeed indicators, pointers on the compass). Some things have both 
components, such as airspeed and compass displays. However, one or both classifications may be 
used depending on the activity. Reading airspeed would be verbal, checking for trend information - 
increase/decrease - would be spatial. In the general case, activities requiring specific state 
information probably are verbal, and those activities requiring trend, or rough estimates, probably 
are spatial. 

Separable versus Integral : These two attributes classify the relatedness of the visual stimuli. 
Relatedness is defined with respect to function. The integral stimuli are related; separable stimuli 
are unrelated. The degree of relatedness depends on the functional nature of the stimuli and are 
defined in relation to the activity, display or environment, or relationships among the stimuli (e.g., 
color). The stimuli may be inherently related (similarities inherent to the stimuli, such as color or 
form), or they may be related by design principle or functionality (different forms but similar 
function, as in the HUD). In either case, related stimuli can be similar displays, similar symbols 
used in displays, similar environmental objects, similar display fonts, and similar colors or color 
codes. Unrelated stimuli are just the opposite: dissimilar displays, symbols, etc. 

Planned versus unplanned : These two classifiers pertain to attentional shifts associated with 
expected and unexpected events. If a stimulus is expected, attention can be primed for the onset of 
the stimulus and can be readily shifted to the stimulus without cost. The shift can occur in 
conjunction with attention focused on the current stimulus or attention can be rapidly shifted 
between the current and expected stimulus without interfering with the processing of the current 
stimulus. If the stimulus is unexpected, the attentional shift can incur a cost and interfere with 
either or both the current and unexpected stimuli. The interference can be in the form of reordering 
the stimuli for processing, restructuring the type of processing, or slow memory access. It can 
cause processing to go from an automatized or perceptual-response mode to an inferential, 
controlled, or "deeper" processing mode (i.e. pattern recognition to problem solving). 

Motor: Each motor taxonomic classifier (e.g., verbal, spatial, gross, fine) is a motor-response 
attribute. The attributes in the motor dimension characterize the action that is required to generate a 
response to a stimulus or set of stimuli (an activity or set of activities). These actions will be 



referred to as the motor responses. The attributes classify the different aspects of a response(s). 

The motor attribute definitions are more straightforward than the others since they classify 
objective, observable behaviors: the specific types of movements, but not the physical effectors of 
movement. The specific types of movements the motor attributes classify can vary in degree, 
requiring classifications based on context 

Verbal versus Spatial : These attributes classify the specific type of response. If the response is 
spoken, it’s verbal. If the response is manual, it's spatial. 

Near versus Far : These attributes classify the extent of the reach required to generate a response. If 
no reach is required or the reach is very-short, it's near. If the required reach is longer than very- 
short, it's far. Since these attributes represent a binary classification of a short-long continuum, 
near represents hands on the stick/collective, far represents a reach to the panel or from one side of 
the panel to the other. The point is to grossly differentiate between reaches that take less time 
versus more time. These attributes are used only with a spatial response, a near/far verbal 
distinction is meaningless 

Discrete versus Continuous : These attributes differentiate between responses requiring a single 
movement or multiple movements. If one movement is required, it s discrete. If more than one 
movement, or the same movement over a length of time is required, it's continuous. For example, 
flipping a switch on the collective/cyclic or a panel is discrete, operating the collective or cyclic is 
continuous. Some movements, such as using a multi-function display, may be either discrete or 
continuous depending on the number of movements. A strict classification would limit one key 
punch for a discrete classification, but practically, a couple of quick key punches could be 
considered discrete. Extensive key punching would be continuous. The operating definition is the 
total time the movement must be maintained, short time periods as opposed to long ones. 

Gross versus Fine : These attributes differentiate between two coarse levels of motor control, and 
gain of a control movement. If a movement requires very slight motor control inputs, it's fine. If a 
movement requires large motor inputs, it's gross. For example, a high gain system would be fine 
and a low gain system would be gross. Using a coolie hat with the OORT for target acquisition or 
low level flight control would be fine motor control, whereas NOE flight control or using the flight 
controls and the HUD to acquire a target with the reticle would be gross motor control. 



4.0 TASK DEMAND AND CONFLICT MATRICES 

This section lists the task attribute demand and conflict matrices and their values. Subsets of the 
values from each of these matrices are used to calculate the load values for single and concurrent 
tasks. In all the matrices that follow, the labels at the top of each column are abbreviations of the 
attribute names for the relevant dimension, For within-dimension matrices the letters match the 
names in the left column. For between-dimension matrices, the letters match the attribute names for 
the dimension listed at the begining of the row. 

4.1 Matrices of Task demand Values: Tables 4-1 to 4-8. 


Table 4-1: Visual Demand Values . 
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Table 4-2: Auditory Demand Values 
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Table 4-3: Cognitive Demand Values 
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Table 4-4: Motor Demand Values. 

(For this dimension, numbers are added to the abbreviations listed at the top to indicate which 
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Table 4-4 continued: Motor Demand Values. 
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Table 4-6: Visual-Cognitive Demand Values 
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Table 4-8: Motor-Cognitive Demand Values . 

(For this dimension, numbers are added to the abbreviations listed at the top to indicate which 
fingers are used, and correspond to the numbers of the fingers listed in the left columns.) 
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4.2 Matrices of Task Conflict Values: Tables 4-9 to, 4-16. 


Table 4-9: Visual Conflict Values . 
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Table 4-10: Auditory Conflict Values 
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Table 4-12: Motor Conflict Values. 
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Table 4-12 continued: Motor Conflict Values. 
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Table 4-13: Visual- Auditory Conflict Values. 
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Table 4-15: Auditory-Cognitive Conflict Values . 
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Table 4-16: Motor-Cognitive Con flict Values ♦ 
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5.0 THE TASK LOADING MODEL CALCULATION ALGORITHMS 


The VACM classifications determine which activity demand and conflict values are passed to the 
load calculation algorithm; This algorithm was derived from one developed by North & Riley 
(1989) and by Wickens & Andre (1989). It multiplies the appropriate demand and conflict values, 
summing their products to generate the load estimates for each dimension: Ly, L^, L<^, and Ly/p 

5.1 The Algorithm’s Factors 

The load calculation algorithm pairs respective attributes of two index lists to select demand values 
from each of the conflict matrices. Three factors comprise this algorithm: VACM Index 
Integration, Activity Demand Integration and the Matrix Combining algorithm. 

5.1.1 VACM Index Selection 

VACM Index Integration integrates indices across tasks to determine which classification indices 
are paired to access their associated activity demand and conflict values. The 4 VACM sets of 
classification indices for each task combine within and across tasks to form the pairs of indices that 
act as the row and column indices to each of the demand and conflict matrices. This algorithm 
computes all the possible pairs of indices between two sets of classification indices. 

For example. 

The visual set for task one maps to visual set for task two. 

(13 5) maps to (2 4 6) 

The mapping forms 18 pairs of matrix reference lists; 

(1 2), (1 4), (1 6), (3, 2), (3, 4), (3, 6, (5 2), (5 4, (5 6) and 
(2, 1), (2, 3), (2, 5), (4, 1), (4, 3), (4, 5), (6, 1), (6, 3), (6, 5) 

to calculate the load value for the interactions within the visual dimension across tasks. 

This procedure repeats to calculate each of the four within dimension load values and each of the 
four between dimension load values that are averaged to calculate the VACM load values for each 
task and set of tasks. 

5.1.2 VACM Index Integration 

VACM Index Integration either integrates index lists across tasks to prevent task order effects, or it 
keeps the index lists separate. Integral sacrifices conficts to reduce the order effects. Separable 
represents all conflicts, but at the price of potentially biasing load estimates due to the order the 
tasks are passed to the algorithms.. 

For example. 

The visual set for task one maps to visual set for task two. 

(13 5) maps to (2 4 6) 

The mapping forms one classification list — (1 2 3 4 5 6). 
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5.1.3 Activity Demand and Conflict Integration 

Activity Demand Integration combines the activity demand values by multiplying the demand 
values in each subset selected from each matrix. 

The product algorithm multiplies the demand values to penalize activity(s) to the extent that the 
interacting attributes have high mutual conflicts (Wickens & Andre, 1989). Multiplying high 
demands adds more to the resulting load values than multiplying low demands. 

5.1.4 Matrix Combining Algorithms 

Average across within and between matrices for each dimension. 

5.2 Assumptions Underlying the Algorithms 

Three assumptions were made to ensure that the algorithm was mathematically fair across the order 
of attributes. 

1) A set consists of four values that defines a unique relationship between the rank orders of two 
pairs of attributes (two pairs of indices). The order of the values is dependent on the interactions 
among the two pairs, and only one value from the set enters into the algorithms. 

2) The values used in the Activity Demand Integration factor are cross-multiplied, they do not have 
a unique relationship among each other and are, therefore, independent of order. 

3) The values from each set are independent, therefore, the algorithms of the first factor insure that 
any one value is not given extra weight. 

5.3 VACM Index Integration Algorithm for Calculating each Matrix Load 

There are two distinct cases for calculating each matrix load: (1) Single Activities and (2) Multiple, 
Concurrent Activities. First, the pairs of indices are computed. Then the indices are used to access 
the demand values in the demand and conflict value matrices; the demand values are then mulitplied 
by its respective conflict value representing the conflict penalty. The Activity Demand Integration is 
represented by (d^d^) (c^c^) in the equation below. 

Ml = X FI FI .FI (d^d^) ( c tki c ukP 
t=l u=t+l k=l i=l 

dtki = indices to the rows in the demand matrices 
d u ki = indices to the columns in the demand matrices 
c tki = indices to the rows in the conflict matrices 
c uki = indices to the columns in the conflict matrices 
s = number of concurrent activities 
n = number of rows 
1 = number of columns 
t, u = activity indices 

5.4 Matrix Combing Algorithms to Calculate each VACM load 

The final load value for each dimension is computed by averaging the within-matrix value with the 
appropriate between-matrix values. Appropriateness depends on which dimensions interact. 


Information is assumed to be processed serially across stages. Consequently, the visual and 
auditory dimensions (the input modalities) interact with the central processing dimension (cognitive 
dimension) since they feed in the information and operate concurrently. The central processing 
dimension interacts with motor performance (the motor dimension) since it determines and 
monitors the course of action. There is no direct interaction between motor and vision or auditory. 
These interactions occur indirectly via cognition. 

5.4.1 Visual Dimension 


L V = V cw + [ (VA bw + VC bw) /2 ] 

V cw = loading value for the visual within-matrix 

VA bw = loading values for the visual -auditory between-matrix 

VC bw = loading values for the visual-cognitive between-matrix 

5.4.2 Auditory Dimension 

L A = A cw + [(VA bw + AC bw )/2] 

Ac W = loading value for the auditory within-matrix 

VAbw = loading values for the visual-auditory between-matrix 

AC bw = loading values for the auditory-cognitive between-matrix 

5.4.3 Cognitive Dimension 


L C ^cw + [^^bw + ^^bw + ^^bw^ ^ 2 ] 

C cw = loading value for the cognitive within-matrix 
AC bw = loading values for the auditory-auditory between-matrix 
VCbw = loading values for the visual-cognitive between-matrix 
CM bw = loading values for the cognitive-motor between-matrix 

5.4.4 Motor Dimension 


Lm ^cw + ^^bw 

M cw = loading value for the motor within-matrix 

CM bw = loading values for the cognitive-motor between-matrix 
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6.0 TLM VALIDATION PARAMETERS 

The following sections list and discuss the parameters tested to validate the TLM. 

6.1 TLM Base Experimental Design 

Figure 6-1 shows the factors comprising the base design of the TLM that was implemented and 
tested. The TLM was tested by correlating load estimates for experimental conditions with the 
empirical data for those conditions. These factors are the same as shown in Figure 1-1, the 
overview of the TLM. 



Figure 6-1: TLM Base Design 


6.2 Experimental Design Parameters 

Figure 6-2 shows the parameters that were varied for each of the factors in the basic design, which 
are discussed in the following sections. 



DEMAND VALUES 

Figure 6-2: TLM Base Design Parameters 
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6.2.1 Taxonomy 


The taxonomy is comprised of sets of binary task classification attributes for each of the four 
VACM dimensions. 

6.2.2 Demand Values 

Demand Values are sets of rank-ordered demands 

6.2.2. 1 Demand Matrices 

Demand matrices are matrices of the ranks assigned to the task attributes. These values represent 
the demands impacting an operator during task performance. 

6. 2. 2. 2 Demand Matrix Values 

The values in the demand matrices vary across task attributes represent varying magnitudes in 
demand. 


6.2.3 Conflict Values 

6. 2. 3.1 Global Conflict Values 

Conflict values are the penalties assigned to the interacting demands that conflict during task 
performance. All demands are multiplied by a single global conflict value that represents the 
amount of conflicting resources. As the number of conflicts increase across tasks, the penalties 
increase. 


6.2.4 Algorithms 

6.2.4. 1 VACM Index Selection 

This factor determines how the task attribute index pairs that index the matrices of demand and 
conflict values are selected from the lists of task classifications. 

6. 2. 4. 2 VACM Index Integration 

This factor determines how the task attribute lists are combined across tasks. 

6. 2. 4. 3 Activity Demand and Conflict Integration 

This factor determines the type of calculations used in the equations that combine the demand 
conflict values. 

6. 2. 4. 4 Matrix Combining Algorithm 

This factor determines how the values from the calculations for the within and between matrices are 
combined across dimensions. 

6.3 Experimental Variations of the Design Parameters 

Questions arose while developing the TLM. The questions led to variations in the design that 
needed experimental verification. Figure 6-3 lists the variations. 

6.3.1 Taxonomy 


27 



6. 3. 1.1 Full Set: 


Table 3-1 


Attribute Pairs per dimension: 5 4 5 4 

Visual: near/far, scan/fixate, s-n ratio, salient/masked, static/dynamic 

Auditory: orient/discriminate, signal/speech, s-n ratio, temporal/physical location 

Cognitive: direct/transform, single/multiple choice, verbal/spatial, separable/integral, 

planned/unplanned 

Motor: verbal/spatial, near/far, discrete/continuous, gross/fine 

Use Table 3-2 to classify activities using this full set of attributes. 



Figure 6-3: Variations in the Design Parameters That Were Tested 

6.3. 1.2 3 Pair Set: Table6-1 

Attribute Pairs per dimension: 3 3 3 3 

Visual: near/far, scan/fixate, salient/masked 

Auditory: orient/discriminate, signal/speech, temporal/physical location 

Cognitive: direct/transform, single/multiple choice, planned/unplanned 

Motor: verbal/spatial, discrete/continuous, gross/fine 

Use Table 6-1 to classify activities using this full set of attributes. 
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Table 6-1: 3 pair taxonomic classification set spreadsheet. 
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Table 6-3: 3 pair taxonomic classification set spreadsheet. 
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6.3. 1.3 2 Pair Set: 


Table 6-2 

- i 

% 

Attribute Pairs per dimension: 2 2 2 2' 

Visual: scan/fixate, salient/masked 

Auditory: orient/discriminate, temporal/physical location 

Cognitive: direct/transform, single/multiple choice 

Motor: verbal/spatial, discrete/continuous 

Use Table 6-2 to classify activities using this full set of attributes. 

6.3.2 Demand Values 

6.3.2. 1 Demand Matrices 

Eight: Four within-dimension and four between-dimension matrices. 

Demand matrices are matrices of the ranks assigned to the task attributes. These values represent 
the demands impacting an operator during task performance. 

6. 3. 2. 2 Demand Matrix Values 

Range: 0-1, 0-4, or 0-10 

The test here is that a wider range of values better represent varying magnitudes in demand. Based 
on this assumption, the 0- 1 range should correlate least well, the 0-4 range should correlate better, 
and the 0-10 range should correlate best. 

However, Wickens' research, suggests that a binary set of demand values. Range 0-1, will add 
more systematic than random variance and correlate better than wider ranges, because they will 
only add more random than systematic variance. 

6.3.3 Conflict Values 

6.3.3. 1 Global Conflict Values 

Values: 1,2, or 3 

This factor globally increases the penalty value for conflicts among activities as the number of 
activities increases, irrespective of the type of conflict. 

6. 3. 3. 2 Conflict Matrices 

Eight: Four within-dimension and four between-dimension matrices. 

Conflict matrices are matrices of the penalties assigned to interacting task attributes that conflict 
during task performance. The number of matrices that best represent the set of weights must equal 
the number of demand matrices, since each demand matrix must have an associated set of weights. 
Visual-motor and auditory-motor weights won’t be assigned if they do not have associated demand 
matrices. 
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6. 3. 3. 3 Conflict Matrix Values 

Range: 0-10 



Multiplying demand values by conflict values specific to the conflicting demands represents 
conflicting resources between all tasks better than multiplying all demands by a single conflict 
value that represents the amount of conflicting resources. 

6.3.4 Algorithms 

6.3.4. 1 VACM Index Selection: 

Three algorithms were tested to determine how the task attribute index pairs should be selected 
from the lists of task classification indices entered into the task classification spreadsheets. 

All combinations across tasks : Separately selects from each concurrent task all of the possible 
pairwise combinations among the classification lists for each task. This does not capture conflicts 
between tasks. 

Sequential pairings across tasks : Separately selects from each concurrent task the sequential pairs 
from the classification lists for each task. This is the simplest case for combining the activity 
demand and conflict values. This does not capture conflicts between tasks. The order of the tasks 
does not affect the load estimates, because the estimates are calculated for each task and then 
summed across tasks. 

Permutations across tasks : Selects the possible pairwise permutations between the classification 
lists of concurrent tasks. This captures conflicts between tasks. The order of the tasks does not 
affect the load estimates, because the permutations capture both of the orders possible for matrix 
reference. 

6. 3. 4. 2 VACM Index Integration 

Two algorithms were tested to determine how the task attribute lists should be combined across 
tasks: Integral and Separable. 

Integral: One classification list for each dimension is created by taking the union of each 
dimension’s lists across all of the tasks. Pairs of indices are selected from each of the four 
integrated lists. 

Separable: The classifications lists for each dimension are kept separate across tasks. Pairs of 
indices are selected from all of the lists. 

6. 3. 4. 3 Activity Demand and Conflict Integration 

Operand: Sum, Average or Product 

This factor determines the operand applied to the demand and conflict values for each within- and 
between-dimension matrix. 

6. 3. 4. 4 Matrix Combining Algorithms 

Operand: Sum or Average 

Sums or Averages across the load estimates for the within- and between-dimension matrices to 
calculate the load for each VACM dimension. 



TAXONOMY 


6.4 Final Design Parameters 

Figure 6-4 lists the final design parameters discussed in the previous sections. 



DEMAND VALUES 


Figure 6-4: TLM Final Design Parameters 
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7.0 SUMMARY OF VALIDATION TESTS 


Based on results from 4 tests, the TLM parameters that best model the data have been integrated 
into MIDAS. This section summarizes the results. A detailed discussion of the results is presented 
with the data in the next section. 

The TLM’s parameters were tested in two separate analyses of data from four experiments. In the 
first analysis, all of the parameters were tested. In the second analysis, a subset of the parameters 
were tested that included some changes based on the results of the first analysis. In the second 
analysis, the data from experiment 1 were not modeled again because the data were too variable to 
be reliable. 

7.1 First Analysis of All Parametric Variations 

All of the parameters that were varied were tested by fitting the TLM load estimates for conditions 
from four experiments to the results of those experiments. 


Table 7-l:Parameters Tested 


Taxonomy 

Taxonomy Attribute Pairs 

Demand Values 

Demand Value Matrices 
Matrix Values 

Conflict Values 

Global Conflict Values 
Conflict Value Matrices 
Matrix Values 

Algorithms 

VACM Index Selection 
VACM Index Integration 
Activity Demand & Conflict Integration 
Matrix Combining Algorithms 


All of the variations are not listed here because Figure 6-3 lists them. The variations resulted in 324 
different load estimates for each condition that were generated from 324 different sets of 
parameters: each set contained one variation of each parameter from each factor listed above. 

NOTE: Three variations listed in Figure 6-3 were not used in the first analysis, because they were 
added as a result of the first analysis. They are VACM Index Selection: Permutations Across 
Tasks, Conflict Values: Conflict Value Matrices, and Matrix Values. 

7.1.1 Test 1: A rough look at the activity integration factor: using integral vs 
separable parameters with a full classification set to predict workload in a military 
helicopter environment. 


This test only generated two sets of load estimates. 


Demand Values: 

Algorithms: 

Demand Value Matrices: 

VACM Index Selection: 

8 

All Combinations Within Tasks 

Matrix Values: 

VACM Index Integration 

0-4 

Integral 

Separable 

Activity Demand & Conflict Integration: 
Product 

Matrix Combining Algorithm 
Average 
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Results indicated that predicted loads were sensitive to how activity classification indices represent 
activity conflicts. Load estimates generated from separate dual task classifications correlated 
marginally higher than from integrated (the union of all classifications) dual task classifications 
(Tables 7-3 & 7-4). This was felt to result from the fact that more conflicts were captured using 
dual classifications than were captured using integrated classifications. Conflicts were represented 
by multiplying demand values together; multiplying larger demand values implicitly factored in a 
conflict penalty. 


Table 7-3: 

Correlations between the paired comparison data from the attack and scout pilots and the MIDAS 
TLM predicted task loads (V=Visual, A=Auditory, C=Cognitive, M=Motor, Avg=Average Of 
VACM Values, Sc=Separable Classifications, Ic=Integral Classifications, *=Significant at the 0.01 

level) 


1 

V 

A 

C 

M 

Avg 

Attack 

Single tasks 

0.5782 

-0.3958 

0.3677 

0.1780 

0.4024 

Dual tasks - SC 

0.3682 

-0.5966 

-0.1092 

0.3537 

-0.4105 

Dual tasks - CC 

0.3036 

-0.3826 

-0.2192 

-0.1006 

-0.0682 

Scout 

Single tasks 

0.6077* 

-0.4363 

0.6401* 

0.5402* 

0.5831* 

Dual tasks - SC 

0.0329 

0.0106 

0.5424 

0.2325 

0.4123 

Dual tasks - CC 

-0.1465 

-0.2069 

0.0519 

0.4227 

-0.1232 


Table 7-4: 

Correlations between the workload ratings from the Attack and Scout pilots and the MIDAS TLM 
predicted task loads (V=Visual, A=Auditory, C=Cognitive, M=Motor, Avg=Average of VACM 
values, SC=Separable Classifications, IC=Integral Classifications, *=significant at the 0.01 level) 



V 

A 

C 

M 

Avg 

Attack 

Single tasks 

0.5859* 

-0.3595 

0.4814 

0.1868 

0.4780 

Dual tasks - SC 

-0.0208 

-0.0836 

0.4336 

0.0992 

0.1431 

Dual Tasks - IC 

-0.0993 

-0.2849 

0.2262 

0.4851 

-0.1059 

Scout 

Single tasks 

0.4582 

-0.3440 

0.6789* 

0.5432 

0.5324 

Dual tasks - SC 

-0.1999 

0.1028 

0.7079* 

0.1484 

0.4719 

Dual tasks - IC 

-0.007 

0.1049 

0.3344 

0.4631 

0.1455 


Since this experiment indicated that capturing conflicts in addition to demands was important, 
conflict and conflict values were separated from the demand values to explicitly try to capture their 
effect. Consequently, tests 2, 3, and 4 represent conflicts as a separate Conflict Value Factor. 

Another finding was that the dual task classifications were confounded with order; The order of 
classification lists among the tasks determines the row and column values for referencing the 
demand and conflict matrices. This order effect potentially can affect the load estimates because 
some interactions are not captured that may be important. 

Because of the order effect and the fact that the marginal correlations may have stemmed from 
methodological problems in data collection, integrated classifications were used to generate load 
estimates for tests 2, 3 and 4 for the first analysis. However, the Integral and Separable variations 
were tested in the second analysis. 




7.1.2 Test 2: Correlating predictions between the TLM, and the TLAP, VACP 
and W/Index workload models in a multiple task setting. 

Table 7-5 lists all the parameters and variations in test 2 that were compared to the data from 
Experiment 2. (These were also tested against data from Experiments 3 and 4.) 

Table 7-5 


Parameters and Variations Tested 


Taxonomy 

Algorithms: 

Taxonomy Attribute Pairs: 

VACM Index Selection: 

Full set 

All Combinations of Tasks 

3 Pair Set 

Sequential Pairing of Tasks 

2 Pair Set 

VACM Index Integration: 
Integra] 

Demand Values: 

Activity Demand & Conflict Integration: 

Demand Value Matrices: 

Sum 

8 

Average 

Matrix Values: 

Product 

0-1 

Matrix Combining Algorithm: 

0-4 

Sum 

0-10 

Conflict Values: 

Conflict Value Matrices: 
8 

Matrix Values: 

0-10 

Average 


7. 1.2. 3 Task Conditions 

The tasks used in this experiment were coded and classified according to the TLM Taxonomy, and 
used to predict the task demands imposed on the operator by the 17 task conditions. These 
predictions will then be correlated to the data used in the Samo and Wickens (1991) study. 

Task Conditions: 

1) Tracking task 

2) Monitoring task 

3) Decision tasks 

1) visual or auditory presentation 

2) spatial or verbal cognition 

3) easy or hard conditions 

4) key press or voice responding 

7. 1.2. 4 Subtasks 

These conditions were evaluated using a task analysis to determine the list of subtasks the subjects 
had performed (Samo & Wickens, 1991; pp 24-25). Seven subtasks were delineated and were 
assigned demands for each of the workload components in the three different models: TLAP, 
VACP and W/INDEX. These models were then used to make predictions for each of the sixteen 
multiple task conditions using the appropriate combinations of the seven subtasks. Product 
moment correlations were calculated between these predictions and the mean tracking decrements 
for each condition. 
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The seven subtasks used are 


Tracking Visual Processing Spatial Cognition Manual Responding 
Speech Processing Verbal Cognition Voice Responding 

7. 1.2. 5 Contrasting the TLM with TLAP, VACP, and W/INDEX 

Samo & Wickens (1991) reported that their decision task data demonstrated little variance in 
performance as a result of multiple task interference. Therefore, they calculated percent tracking 
decrement scores for each of the 16 conditions based on decomposing the conditions into their 
respective subtasks, listed above. These tracking decrements were used to fit the predictions from 
the workload models. The TLM predicted loads for each of the 16 decision tasks paired with the 
tracking task. The TLM predictions used the same procedure to generate predictions that the 
workload models used (see section 9-11). The W/INDEX, VACP, and TLAP all produced one 
load estimate, so the individual TLM VACM load estimates were averaged for a comparable 
comparison. However, the mean VACM value showed the same patterns of results, although the 
correlations were lower than its component load estimates. Therefore, the component VACM 
values were graphed because they are more diagnostic, and because the TLM was designed to 
produce estimates along each dimension. 

7. 1.2. 6 Results 

The numbers on the x-axis of figures 7-1 to 7-5 correspond to the different algorithms used to 
calculate TLM load predictions; They are composed of the different combinations of parameters 
listed in Table 7-5. Table 7-6 maps the parameters to algorithm number only for the Full-set of 
taxonomic classifiers, because the same parameteric combinations repeat for the other sets of 
attribute pairs. 


7. 1.2. 6.1 Taxonomy Factor 

Figures 7-1 and 7-2 show the same patterns of correlations between the TLM load estimates and 
the Tracking decrement scores for the full-set and 2-pair set of taxonomic classifiers. Figures 7-3, 
7-4 and 7-5 show the same patterns of correlations between the TLM and the TLAP, VACP and 
W/INDEX models. These patterns were the same for the 3-pair set of taxonomic classifiers, so 
these data weren’t displayed. These patterns clearly show that the different levels of task attribute 
detail used to classify the tasks didn’t have an effect. Parsimony would suggest rejecting the full- 
set and the 3-pair set. However, the full set of task attributes was kept as a final parameter because 
1) It allows for more detailed classifications, which can potentially account for more conflicts 
among tasks and because the Full-set is currently in use, and 2) This factor wasn’t sufficiently 
tested in this analysis, because the experimental conditions didn’t methodically manipulate the level 
of task attribute detail. The final test (Experiment 5) examines the level of task detail required more 
thoroughly. If the results from this next test show no difference, then the number of task attributes 
TLM uses should be reduced. 

7. 1.2. 6. 2 Demand Value Factor 

The three redundant patterns Figures 7-1 and 7-2 show results from the three Matrix Demand 
Value parameters. Matrix Values 0-10, 0-4, or 0-1. These patterns correspond to algorithms 1-18, 
19-36, and 37-54 as shown in Table 7-6. Since the three patterns are the same, two of the Matrix 
Values can be rejected. For parsimony, 0-4 and 0-10 were rejected. The workload data shown in 
Figures 7-3, 7-4 and 7-5 clearly show the same patterns. 



Table 7-6: The combinations of parameters tested that correspond to the 
Algorithm Type number list on the x-axis of Figures 7-1 to 7-5. 


Algorithm 
Type # 

Taxonomy 

Matrix Demand 
Value 

Activity Index 
Selection 

Activity Demand & 
Conflict Integration 

Conflict Penalty 
Value 

1-3 

Full Set 

0-10 

Combination 

Average 

1,2,3 

4-6 




Sum 

1,2,3 

7-9 




Product 

1,2,3 

10-12 



Sequential 

Average 

1,2,3 

13-15 




Sum 

1,2,3 

16-18 




Product 

1,2,3 

g&s ; v 


0-4 

Combination 

Average 

1,2,3 

W5ESSM 




Sum 

1,2,3 

25-27 




Product 

1,2,3 

28-30 



Sequential 

Average 

1,2,3 

31-33 




Sum 

1,2, 3 

34-36 




Product 

1,2,3 

37-39 


0-1 

Combination 

Average 

1,2,3 

40-42 




Sum 


43-45 




Product 

KffilHi 

46-48 



Sequential 

Average 

1,2,3 

49-51 




Sum 

1,2,3 

52-54 




Product 

1,2,3 


7. 1.2. 6. 3 Conflict Value Factor 

In this first analysis, only the global conflict value parameter was used. Figures 7-1 to 7-5 show 
that increasing the value of the global conflict penalty generally increases the correlations. This can 
be seen by examining the algorithms in sets of three, e.g., 1-3, 4-6, ... 52-54. These triples map to 
conflict values of 1, 2 and 3 respectively. In most cases, the correlations increase as the conflict 
values increase, although the amount of increase varies widely across algorithms. The consistently 
higher correlations associated with higher values indicates that the conflict values can substantially 
affect the load estimates, often increasing the correlation coefficient by 0.2, except for algorithms 
16-18. In these cases, the coefficients jump substantially from conflict values of 1 to 2, up to 0.7. 
They remain the same from 2-3 indicating that conflicts aren’t discriminated further using a global 
increase. 


38 














l.O-i 


Full Taxonomy: VACM with tracking 


■o visual 



Algorithm Type 

Figure 7-1: Correlations of TLM Load Predictions with percent tracking 
decrement scores using the full set of task attribute classifiers. 

These results show that conflict values 1 and 2 can be rejected, using only a global value of 3. 
However, this approach can also be modified to try and increase the correlations by making the 
values more diagnostic. This approach was selected, but not applied to all the algorithms because 
of the repetitive effects. Choosing a more diagnostic approach was selected for testing in the 
second analysis because the results show the impact from penalizing conflicts, even when the 
penalties are applied indiscriminately to conflicts between demands, i.e., irrespective of the type of 
demands. Algorithm 18 was selected because it has the highest correlations, and because it didn’t 
increase from values 2 to 3 implying a ceiling effect that could be mitigated using more diagnostic 
approach. 
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Level 2 Taxonomy: VACM components with tracking 



Algorithm Type 

Figure 7-2: Correlations of TLM Load Predictions with percent tracking 
decrement scores using the set of task attribute classifiers reduced to two attribute 

pairs for each dimension 


7. 1.2. 6. 4 Algorithm Factor 

7. 1.2. 6. 4.1 Algorithm Index Selection 

This parameter was varied by manipulating the way pairs of indices are formed. The Figures show 
that Algorithms 1-9, forming index pairs from all possible combinations of classification lists that 
are integrated across tasks, correlated less well than Sequential Selection, Algorithms 10-18. These 
patterns were repeated for the rest of the algorithms. This is a surprise because combinations 
capture all possible conflicts between demands, whereas sequential pairing does not. In fact 
sequential pairing captures only a small number of the conflicts. This suggests that capturing all 
possible conflicts adds noise past a certain point. The addition of conflicts past this point could add 
noise because relevant and irrelevant conflicts aren’t distinguished. The best representation of 
conflicts then is not the total of them, but a set of important ones. In this light, combinations add to 
many relevant and/or irrelevant conflicts, while sequential pairing adds to few. 


40 



o visual 



Figure 7-3 Correlations of TLM Load estimates with TLAP estimates using the 

full set of task attribute classifiers. 


The first analysis could not distinguish between relevant or irrelevant conflicts because the index 
lists were integrated before the pairs were formed. This completely confounded the different types 
of conflicts among demands between tasks. This finding supports the results from Test one, which 
indicated that lists of indices that were integrated across tasks correlated less well than separated 
lists. Since the integrated lists were chosen for this first analysis to mitigate a possible order effect 
in the light of the marginal correlations from Test 1, a different approach was chosen for analysis 
two that uses separable classification lists that account for any order of tasks. This approach uses 
permutations of classification lists across tasks, which avoids order effects and also differentiates 
between relevant and irrelevant conflicts between task demands. 

7. 1.2. 6. 4. 2 Activity Demand and Conflict Integration 

This parameter determines whether the products of the demands and the conflict values are 
averaged, summed or multiplied. The Figures show that the average correlates least well, and that 
the product correlates best. The sum correlations are highest for Sequential Index Selection. 
Though not as high as the product, they were sufficiently high to retain for the second analysis. 



Full Taxonomy: VACM with VACP 


visual 

auditory 

motor 
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Figure 7-4 Correlations of TLM Load estimates with VACP estimates using the 

full set of task attribute classifiers. 

7. 1.2. 6. 4. 3 Matrix Combing Algorithms 

This parameter determines how the values that are calculated for each of the within- and between- 
matrices using the Activity Demand and Conflict Integration parameter are combined. They were 
either summed or averaged. There was no difference between the two, so the Average was chosen. 

7. 1.2. 6. 5 Correlation coefficient comparisons between the Models 

The figures indicate algorithm type 18 generates the highest correlation. It uses the full taxonomic 
set of task attribute classifiers, task demand values ranging from 0 to 10, the summation and 
product algorithms, and a conflict penalty value of 3. 


Full Taxonomy: VACM with W/INDEX 


visual 

auditory 

cognitive 



Algorithm Type 

Figure 7-5 Correlations of TLM Load estimates with W/INDEX estimates using 

the full set of task attribute classifiers. 


Table 7-7 lists the correlations between the load values predicted using #18, the predictions from 
the other models and the tracking decrement scores. It shows that the TLM overall load estimate 
correlates with tracking decrement scores at least as well as the TLAP and VACP models, but less 
well than W/INDEX. The TLAP, VACP and W/INDEX models only produced one load estimate 
in this experiment, so the TLM VACM load estimates were averaged for a comparable comparison. 
This overall load estimate is listed in the row titles TLM Overall Load. 

However, the coefficients for the individual VACM dimensions are more diagnostic, and need to 
be considered. The highest correlations are found between for the cognitive and motor dimensions. 
This finding supports the diagnostic claim. The tracking decrement scores were for concurrent 
tracking and decision tasks that theoretically should impose more demands on motor and cognitive 
processes than on visual and auditory processes. Despite the fact that the decision tasks 
manipulated the auditory and visual modalities, TLM estimates did not correlate along these 



dimensions. Since only the VACP model has component estimates and they weren’t reported, 
component comparisons between models can’t be made. 


Table 7-7: 

Correlations between the various model predictions and tracking decrement 
scores. TLM predictions are for New Algorithms 1-6. 

n=16, df=14, p<.10 = .426, p<.05 = .497, p<.01 = .623 


Model 

Tracking 

TLAP 

VACP 

W/INDEX 

W/INDEX NC 

TLAP 






VACP 

.7053 

.8306 




1 'i'll It Ml I 



.8244 



W/INDEX NC: Ho spatial 
or verbal Code 

.8301 

.8645 

.8800 






■■■1 

TLM Overall Load (mean 
VACM) 




.8180 

.8689 

MM 

mu 

HUH 

Visual 

.3761 

.4367 

.7902 

.3594 

.4426 

Auditory 

.0 

.0 

.0 

.0 

.0 

Cognitive 

.6764 

.8334 

.9475 

.82H4 

.8674 

Motor 

.7829 

.7728 

.7330 

.9476 

.9345 


7. 1.2.7 Summary of Test 2 

In summary, the parameters can be classified as sensitive or insensitive to the experimental 
conditions (Table 7-8). The sets of algorithms were reduced to one that generated load estimates 
that correlated fairly well with the tracking decrement scores and with the other models 
predictions. 


7.1.3 Tests 3 and 4: 

Predicting Attention effects 
using TLM in a multi-task 
windowing environment. 

Tests 3 and 4 were conducted on data 
from two experiments that used a low 
fidelity multi-task simulation that 
investigated the effects of cue 
specificity on task preparation and 
performance (Andre and Heers, 1993). 
Both experiments were designed and 
presented using Window/P ANES: Workload/PerformANcE Simulation (Andre & Heers, 1993). 

The first experiment manipulated type of cue, either specific, general or no-cue. The second 
experiment manipulated difficulty level along with type of cue. The TLM was used to generate load 
estimates only for the different cue manipulations in both experiments, not for the difficulty 
manipulations. The TLM does not explicitly represent levels of difficulty. It implicitly represents 
difficulty as a function of the interacting task attributes, some of which impose more demands than 
others. (Experiment 5 was designed to test whether implicitly representing different levels of 
difficulty suffices.) 


Table 7-8: 
Test 2 Results 


Insensitive Parameters 

Sensitive Parameters 

Algorithms: 

Matrix Combining 
algorithms 

Taxonomy 

Taxonomy Attribute Pairs 
Demand Values 
Demand Value Matrices 

Algorithms: 

VACM Index Selection 

Activity Demand & Conflict 
Integration 
Conflict Values 
Global Conflict Values 
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Since the second experiment was an extension of the first experiment, both will be discussed 
together. The TLM comparisons will state which data set is being modeled. 

7. 1.3.1 TLM Parameters Tested 

The parameters tested against the Window/P ANES data, tests 3 and 4, were the same as those 
tested against the Samo & Wickens (1991) data, test 2 (Table 7-5). The results were very similar to 
those found in test 2. The results are discussed in the same order, but no figures will be presented, 
just tables of correlation coefficients. 

7. 1.3. 2 Experimental Tasks and Conditions 

The Window/P ANES software presents a color display with four quadrants (Figure 7-6). The 
upper left quadrant displays alphanumeric messages and presented task questions and cue 
messages. The upper right quadrant displays the target and ownship symbols for the primary 
compensatory tracking task and includes digital heading, altitude, and speed indicators at the 
bottom , which were used in the flight path estimation task. The indicators changed value 
throughout the trial to reflect the ownships current position in each of the axes. The lower left 
quadrant displayed one digital and three analog gauges that were used for the gauge reference task. 
Finally, the lower right quadrant displayed a static top-down map composed of a variety of colored 
symbols that were used for the map orientation task. 

Tracking Task : Subjects performed a 
continuous compensatory tracking task 
throughout the entire trial. This Primary 
task was periodically time-shared with one 
of four secondary tasks: call sign, flight 
path estimation gauge reference and map 
orientation. 


Call Sig n: Subjects memorized a four-item 
alphanumeric string at the beginning of 
each trial for later recognition. Each time 
the message displayed an alphanumeric 
string, subjects had to decided if the string 
was identical to YES, or different from 
NO, the memorized string. 

Flight Path Estimation : Subjects answered yes/no about their current heading or altitude relative to 
a specified heading or altitude by consulting the two indicators at the bottom of the tracking 
window. Questions pertained to either one axis or the other, never both. 

Gauge Reference : Subjects answered yes/no to questions about the status of the red/safe zones of 
one or more of the gauges. The four gauges periodically changed values between their safe and 
upper and lower red zones. From the onset of the gauge question, the gauges were stabilized 
during the remainder of the task response interval to allow subjects to unambiguously assess their 
values. 

Map Orientation : Subjects answered yes/no questions regarding the relative left/right locations of 
two specified map features. Subjects were instructed to view the map window as a static map 
which they were traveling across from left to right. To correctly, answer the question, subjects 
needed to locate the two items and assume the correct left-to-right orientation before judging their 
relative locations. 


Figure 7-6: Window/PANES Display 




Cue Information : Specific cues informed the subject of the type of task about too ccur; General 
cues informed the subject only that a task was about to occur; or No Cues were given. 

7. 1.3. 3 Results 

Thee correlations listed in the tables that follow were generated by separately correlating TLM 
predictions with the performance data (collapsed across subjects) for the four different tasks (n = 

4). The correlations were generated separately for the cue and no-cue conditions (four tasks per 
condition). The performance data correlated with TLM predictions for Andre & Heers’ first 
experiment were rt and rms scores. For the second experiment, rt and rms performance data were 
used as well as four subjective measures: a visual load score (vl), a cognitive load score (cl), an 
overall workload score (ow), and a rank ordering of the difficulty of the tasks (rank). 

7. 1.3. 3.1 Taxonomy Factor 
This factor was not tested. 

7. 1.3. 3. 2 Demand Value Factor 

The three redundant patterns found in Figures 7-1 to 7-5 resulting from the three Matrix Demand 
Value parameters. Matrix Values 0-10, 0-4, or 0-1 also were found in tests 3 and 4. These patterns 
clearly show in the correlations listed in Table 7-9. This table only show results for the cue 
conditions for both Andre & Heer’s (1993) experiments, but the same results were found for no- 
cue conditions as well. This supports rejecting Matrix Values 0-4 and 0-10. Because of the low n, 
no significant correlations were found even though the correlations are high for the visual and 
cognitive dimensions. 

Table 7-9: Window/PANES 1 Cue Conditions 7. 1.3. 3. 3 Conflict Value 

Factor 

Again only the global conflict value was 
used. Table 7-9 shows fairly similar 
correlations for algorithms 17 and 18, 
which used conflict values of 2 and 3 
respectively. Not shown is algorithm 
16, which had very low correlations. 
These results mirror those from test 2. 


7. 1.3. 3. 4 Algorithm Factor 


The correlations of each of the 
algorithms with the Window/PANES 
data sets were high only for the 6 
algorithms listed in the algorithm 
column in Table 7-8. This finding also 
mirrors the findings from test 2. These 
6 algorithms use the parameters 
discussed in the following sections 
(also listed in Table 7-8). 


(Val = Matrix Value, Alg = Algorithm). 


Val 

era 

DV 

DEM 

aud 

EH 

mot 

o w 

0-10 

17 

rms 

.233 

0 

.364 

0 

.269 



rt 

.768 

0 

.882 

0 

.802 


18 

rms 

.269 

0 

.378 

0 

.287 



rt 

.802 

0 

.892 

0 

.818 









0-4 

35 

rms 

.233 

0 

.364 

0 

.269 



rt 

.768 

0 

.882 

0 

.802 


36 

rms 

.269 

0 

.378 

0 

.287 



rt 

.802 

0 

.892 

0 

.818 









0-1 

53 

rms 

.233 

0 

.364 

0 

.269 



rt 

.768 

0 

.882 

0 

.802 


54 

rms 

.269 

0 

.378 

0 

.287 



rt 

.802 

0 

.892 

0 

.818 


7. 1.3. 3. 4.1 Algorithm Index Selection 

Sequential Pairing correlated highest. 
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7. 1.3. 3. 4. 2 Activity Demand and Conflict Integration 

The Product of the demand values multiplied by their respective conflict values correlated highest. 

7. 1.3. 3. 4. 3 Matrix Combing Algorithms 

Little if any difference was found between summing and averaging. 

7. 1.3. 4 Goodness of Fit 

Tables 7-10 and 7-11 below show the TLM estimates correlate fairly highly with the rt’s from the 
first and second Window/P ANES experiments, but only for the cued conditions. The estimates 
show very high negative correlations with Window/P ANES 1 No Cue conditions (Table 7-12) and 
not at all for Window/P ANES 2 No Cue conditions (Table 7-13). Why this occurred is not at all 
clear. For all four tables, n=4, df=2, pc. 10 = .900, pc. 05 = .950, pc.01 = .990. 


Table 7-10: 

Window/PANES 1 Cue Conditions (Val = Matrix Value, Alg = Algorithm). 


Val 

Ale 

DV 

vis 

aud 

cog 

mot 

o w 

0-10 

18 

rms 

.269 

0 

.378 

0 

.287 



rt 

.802 

0 

.892 

0 

.818 


Table 7-11: 

Window/PANES 2 Cue Conditions (Val = Matrix Value, Alg = Algorithm). 


Val 

E&a 

DV 

vis 

aud 

nn 

mot 

o w 

0-10 

18 

rms 

.589 

0 

.487 

0 

.574 



rt 

.829 

0 

.760 

0 

.821 


Table 7-12: 

Window/PANES 1 No Cue Conditions (Val = Matrix Value, Alg = Algorithm). 


Val 

Alg 

DV 

vis 

au 

d 

cog 

mot 

ow 

0-10 

18 

rms 

-.981 

0 

-.989 

-.993 

-.989 



rt 

-.712 

0 

-.747 

-.775 

-.746 


Table 7-13: 

Window/PANES 2 No Cue Conditions (Val = Matrix Value, Alg = Algorithm). 


Val 

sua 

DV 

vis 

aud 

BUI 

mot 

OW 

0-10 

18 

rms 

.084 

0 

.059 

.039 

.061 



rt 

-.074 

0 

-.121 

-.160 

-.119 


7.2 Second Test of All Parametric Variations 

This second analysis only tested variations in four parameters of the Algorithm and Conflict Value 
factors. The first analysis indicated that these parameters were worthy of investigation (Table 7- 
14). The parameters are VACM Index Selection: Permutations Across Tasks; VACM Index 
Integration: Separable; Activity Demand & Conflict Integration: Sum and Product ; and Conflict 
Values: Conflict Value Matrices, and Matrix Values. 

























































All of these algorithms use the separable 
index integration algorithm. Task 
classification lists are not combined before 
the index pairs are formed. All index pairs 
are selected from the individual lists from 
all concurrent tasks. They also use the 0- 1 
Demand Matrix Values and the Full-set of 
task attribute pairs. 

These variations resulted in 6 different load 
estimates for each condition that were 
generated from 6 different sets of 
parameters Table 7-15. These load 
estimates were re-correlated in tests 5-8 
against the same four empirical data sets 
used in tests 1-4. 

Table 7-15: 

The combinations of parameters tested that correspond to the Algorithm number in 

Tables 7-16 to 7-20. 


Table 7-14: 

Parameters and Variations Tested 


Conflict Values: 

Algorithms: 

Conflict Value 
Matrices: 

VACM Index Selection: 

8 

All Combinations of Tasks 

Matrix Values: 

Sequential Pairing of Tasks 

0-10 

Permutations of Tasks 
VACM Index Integration: 
Separable 

Activity Demand & Conflict 
Integration: 

Sum 

Product 


Algorithm 
Type # 

Matrix Demand 
Value 

Matrix Conflict 
Value 

Activity Index 
Selection 

Activity Index 
Integration 

Activity Demand 
Integration 

1 

0-1 

0-4 

Combination 

Separable 

Sum 

2 

0-1 

0-4 


Separable 

Product 

3 

0-1 

0-4 

Sequential 

Separable 

Sum 

4 

0-1 

0-4 


Separable 

Product 

5 

0-1 

0-4 

Permutation 

Separable 

Sum 

6 

0-1 

0-4 


Separable 

Product 


7.2.1 Test 5: A rough look at the activity integration factor: using integral vs 
separable parameters with a full classification set to predict workload in a military 
helicopter environment. 

This data set was not re-analyzed. 

7.2.2 Test 6: Correlating predictions between the TLM, and the TLAP, VACP 
and W/Index workload models in a multiple task setting using the new 
algorithms. 

The results shown in Table 7-16 indicate that algorithms 1 and 6 show the highest correlations 
across all models and tracking decrement scores. Comparing these coefficients to those in Table 7- 
7 shows that motor correlations increased for algorithms 1 and 6, cognitive correlations increased 
for algorithm six; some significant auditory correlations occurred; Visual correlations decreased 
sojjigwhat, and only significantly correlate with V ACP estimates. Based on this test, algorithm 6 
appears best; it has the highest correlations and the most significant ones. 
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Table 7-16: 

Correlations between the workload 
models and tracking decrement scores for 
the 6 new algorithms. 

n=16, df=14, pc. 10 = .426, pc.05 = .497, pc.01 = 


7.2.3 Tests 7 and 8: Predicting 
Attention effects using TLM in a 
multi-task windowing 
environment using the new 
algorithms. 


Tests 7 and 8 show that algorithms 1 and 
6 result in the highest correlations, with 
more significant correlations than the 
other algorithms (Tables 7-17 to 7-20), 
but only with rt, not with rms (same as 
the first analyses). Table 7-17 shows rt 
significantly correlating with visual and 
cognitive dimensions, but low 
correlations with the motor dimension 
despite the presence of the tracking task. 
Its possible that the tracking task was 
simply not hard enough. This is 
plausible since Andre & Heers (1993) 
reported that the type of cue showed no 
main of effect for tracking error in 
experiment 1; The tracking task was not 
difficult enough to benefit from cueing. 
Tables 7-18 and 7-19 support this, with 
little change in the degree of correlation 
or the number of significant ones. 
Although, Table 7-19 shows that 
algorithms 3 and 4, the permutation 
algorithms, correlate better relative to the 
cue condition. 


Andre and Heers (1993) reported a main 
effect of task; Subjects data indicated that 
the tasks varied in difficulty. However, 
the data analyzed in this study were 
collapsed across tasks, so they could not 
be individually analyzed. 


Comparing these results to those 
obtained in the first analysis (Tables 7-10 
to 7-13) shows mixed results. Tables 7- 
19 and 7-20 show high though non- 
significant correlations with the visual 
and cognitive dimensions only for the 
cued conditions in both experiments. 
Table 7-20 shows high, and significant 
correlations between rt and the cognitive 
dimension in algorithms 1, 5, and 6; a 
single high, significant correlation with 
the visual dimension for algorithm 4; and 
cognitive and overall load for rms. Its 
hard to interpret the rms correlations 
because there are so few. However, algorithms 1 and 6 consistently correlate the highest. 


.622 


A, f? 

model 

vis 

aud 

c 9£ 

mot 

OW 


flap 

.434 

-.002 

.747 

,832 

.770 


vacp 

.752 

-.467 

.534 

.666 

.571 

1 

pwdx 

.359 

.031 

.696 

• 875 

.741 


ncwdx 

.431 

-.056 

.683 

.848 

.722 


track 

.405 

-.001 

.639 

.839 

.707 









tlap 

.272 

-.083 

.27 

-.393 

-.04 


vacp 

.671 

-.568 

-.054 

-.480 

-.332 

2 

pwdx 

.249 

-.083 

.145 ; 

.553 

-.186 


ncwdx 

.317 

-.177 

.07 

.582 

-.257 


track 

.286 

-.094 

.245 

-.278 

.017 









tlap 

.315 

-.047 


.275 

.421 


vacp 

.660 

-.545 

.21 

.078 

.18 

3 

pwdx 

.275 

-.07 

.345 

.136 

.287 


ncwdx 

.344 

-.158 

.327 

.125 

.269 


track 

.352 

-.085 

.459 

.409 

.460 









tlap 

.302 

-.041 

.446 

.318 

.396 


vacp 

.623 

-.541 

.162 

.12 

.124 

4 

pwdx 

.264 

-.068 

.316 

.183 

.268 


ncwdx 

.328 

-.155 

.296 

.176 

.246 


track 

.354 

-.085 

.435 

.446 

.437 









tlap 

.335 

-.047 

,714 

.482 

.628 


vacp 

.696 

-.540 

.461 

.275 

.371 

5 

pwdx 

.292 

-.059 

.645 

.432 

.553 


ncwdx 

.364 

-.147 

.622 

.385 

.517 


track 

.355 

-.072 

.632 

.605 

.622 









tlap 

.35 

-.091 

.861 

• 8Q6 

.874 


vacp 

.683 

-.581 

.822 

.735 

.831 

6 

pwdx 

.323 

-.109 

.941 

,965 

.982 


ncwdx 

.372 

-.204 

.941 

,945 

.969 


track 

.354 

-.109 

.753 

,839 

.841 




Table 7-20 shows significant correlations only between the visual dimension and rt and between 
the visual dimension and visual workload report. These showed for algorithms 1, 2, 5, and 6, with 
algorithm 6 recording the highest correlations again. 

Table 7-17: TLM load estimates correlated with Window/PANES 1 data collapsed 

across cue and no-cue conditions 


n=8, dl 

f=6, p<.10 = .622. p<.05 = .707, pc.01 = .8 M 

ESI 

model 

EZES 

aud 

LU9 

mot 

o w 

l 

rms 

.464 

0 

.511 

.342 

.51 


rt 

.7 99 

0 

.545 

.178 

h 








2 

rms 

.311 

0 

.378 

0 

.25 


rt 

.687 

0 

.381 

0 

.429 








3 

rms 

.687 

0 

.527 

.431 

E9*m\ 


rt 

.865 

0 

.343 

.129 

.524 








4 

rms 

.723 

0 

.57 

.364 

.664 


rt 

Kim 

0 

.384 

.054 

.58 








5 

rms 

.485 

0 

.519 

.314 

.503 


rt 

.816 

0 

.547 

.175 









6 

rms 

.382 

0 

.57 

.208 

.434 


rt 

.748 

0 

.710 

.136 

.685 


Table 7-18: TLM load estimates correlated with Window/PANES 1 data for no-cue 

conditions 


n =4, df=2, pc.10 = .900. pc.05 = .950, pc.01 = ,22 fl 


e vm 

Model 

vis 

aud 

EEZ9 

mot 

0 w 

i 

rms 

.503 

0 

.800 

.453 

.683 


rt 

.879 

0 

.998 

.158 

.971 








2 

rms 

.388 

0 

.667 

0 

o 1 


rt 

.801 

0 

.865 

0 

0 








3 

rms 

.689 

0 

.566 

.325 

.717 


rt 

.966 

0 

.455 

-.027 

.774 








4 

rms 

.698 

0 

.593 

.181 

.730 


rt 

.965 

0 

.499 

-.192 

.791 








5 

rms 

.525 

0 

.795 

.383 

.668 


rt 

.890 

0 

.998 

.198 

.967 








6 

rms 

.492 

0 

.738 

0 

.584 


rt 

.866 

0 

.983 

0 

.920 
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Unfortunately, comparisons can’t be made with correlations found for Window/P ANES 2 data in 
the first analysis, because the first analysis separated cue from non-cue conditions, while the 
second analysis collapsed across them. This was due to an oversight of the experimenter. 

Table 7-19: TLM load estimates correlated with Window/PANES 1 data for cue 

conditions 

n=4, df=2, p<.10 = .m p<-05 = .950, pc.Ol = .991) 


Algorithm 

Model 

Visual 

Auditory 

Cognitive 

Motor 

Overll 

Workload 

1 

rms 

.346 

0 

.759 

.774 

.593 


rt 

.831 

0 

.993 

.244 

.951 








2 

rms 

.195 

0 

0 i 

0 

.195 


rt 

.728 

0 

0 i 

0 

.728 








i 

rms 

.714 

0 

.958 

.672 

.994 


rt 

.947 

0 

.528 

.075 

.784 








4 

rms 

.752 

0 

.969 

.558 

.987 


rt 

.950 

0 

.557 

-.055 

.824 








5 

rms 

.372 

0 

.801 

.670 

.555 


rt 

.850 

0 

.992 

.201 

.949 








6 

rms 

.155 

0 

.586 

0 

.169 : 


rt 

.638 

0 

.951 

0 

.571 


7.2.4 Summary of the Second Analysis 


Since there was some doubt about whether algorithm 1 or 6 was better, a regression analysis was 
conducted in hopes that the two could be differentiated based on r-squared values adjusted for 
collinearity among the predictor variables. In these analyses, the VACM estimates were regressed 
on the empirical data in each experiment. 

7.3 Regression Analysis 

7.3.1 Test 9: Regressing on Sarno & Wickens tracking decrement scores 

Table 7-21 shows that algorithms 1 and 6 have very similar adjusted r-squared values. 
Consequently, to decide between the two, the regression coefficients were analyzed (Table 7-22). 
This shows that each of the coefficients’ contribution to r-squared is about the same, a little higher 
for algorithm 6 in fact, but the 2 tailed significance test indicates that only two are significant for 
algorithm 1, while three are significant for algorithm 6. Consequently, though 6 may do better, the 
difference will probably be slight. 




Table 7-20: TLM load estimates correlated with Window/PANES 2 data collapsed 
across cue and no-cue conditions. n=8, df=6, pc.10 = .622, p<.05 = .707. p<.01 = .834 





























































































































































































Table 7-21: R-values of the VACM load estimates regressed on the tracking 
decrement scores from the Sarno & Wiclcens study 


t 

l 

2 

3 

4 

5 

6 

r-sq 

0.854 

0.774 

0.269 

0.274 

0.702 

0.837 

r 

0.924 

0.880 

0.519 

0.523 

0.837 

0.914 

IKUDKEE^H 

0.801 

0.692 

0.004 

0.010 

0.593 

0.777 

res ms 

207.134 

320.959 

1039.53 

1033.29 

424.013 

231.926 

st err est 

14.392 

17.915 

32.241 

32.144 

20.591 

15.229 

f-stat 

16.1 

9.45 

1.02 

1.04 

6.48 

14.1 

num df 

4 

4 

4 

4 

4 

4 

den df 

11 

11 

11 

11 

11 

11 


0.000 

0.001 

0.440 

0.430 

0.006 

0.000 


Table 7-22: Regression Coefficients of the VACM load estimates regressed on the 
tracking decrement scores from the Sarno & Wickens study 


Name 

Reg Coeff 

St Error 

St Coeff 

T-Stat 

2tailsig 

Tolerance 



-36.352 

18.041 

-1.12 

-2.0 

0.06 



visual 

1.9805 

2.9397 

0.39 

0.6 

0.51 

0.03769 

0.0060 

auditory 

1.1922 

3.1398 

0.22 

0.3 

0.71 

0.03669 

0.0019 


-3.7583 

1.8004 

-1.08 

-2.0 

0.06 

0.04927 

0.0576 

motor 

10.289 

1.8861 

1.65 

5.4 

0.00 

0.14441 

0.3935 










E&KHiH 

19.663 

-1.40 

-2.3 

0.04 



visual 

101.35 

37.534 

0.80 

2.7 

0.02 

0.16844 

0.1079 

auditory 

89.730 

54.777 

0.42 

1.6 

0.13 

0.22368 

0.0397 



25.471 


-1.8 

0.09 

0.12225 

0.0492 

| motor 

81.059 

19.757 

1.28 

4.1 

0.00 

0.15109 

0.2492 


7.3.2 Test 10: Regressing on Window/PANES 1 Data 

The selection of the best algorithm is more straightforward for Window/PANES 1 data. Table 7-23 
indicates that 3 and 6 are good, with 3 being slightly better. Since these are for rms scores, which 
didn’t correlate well with TLM load estimates, more weight should be given to the rt results. Table 
7-24 shows that 6 accounts for nearly all the variance, with 3 a close second. These results support 
algorithm 6 as a good algorithm. 








































































































































Table 7-23: VACM load estimates regressed on Window/PANES 1 rms collapsed 

across cue and no-cue conditions 



l 

2 

3 

4 

5 

6 

r-sq 

0.407 

0.159 

0.832 

0.538 1 

0.578 

0.815 

* 

r 

0.638 

0.399 

0.912 

0.733 

0.760 

0.902 


-0.03 


0.707 

0.191 

0.261 

0.676 


27.14 

30.77 

7.662 

21.14 

19.31 

8.455 

st err est 

5.210 

5.547 

2.768 

4.598 

4.394 

2.907 

f-stat 

0.92 

0.48 

6.63 

1.55 

1.83 

5.89 

num df 

3 

2 

3 

3 

3 

3 

den df 

4 

5 

4 

4 

4 

4 


0.509 

0.647 

0.049 

0.331 

0.282 

0.059 


Table 7-24: VACM load estimates regressed on Window/PANES 1 rt collapsed 

across cue and no-cue conditions 



l 

2 1 

3 

4 

5 

6 

r-sq 

0.720 

0.471 



0.777 

0.997 

r 

0.848 

0.686 

0.992 

0.957 

0.881 

0.998 


0.510 

0.260 

0.973 

0.854 

0.610 

0.994 

res ms 

915284.8 

1382230.8 


272340.42 


9643.13 

st err est 

956.705 

1175.68 

220.489 

521.862 

852.945 

98.199 

f-stat 

3.43 

2.23 

88.3 

14.6 

4.66 

450.91 

num df 

3 

2 

3 

3 

3 

3 

den df 

4 

5 

4 

4 

4 

4 


0.132 

0.202 

KIBHi 

0.012 

0.085 

0.000 


7.3.3 Test 11: Regressing on Window/PANES 2 Data 

The selection of the best algorithm is not as clear for Window/PANES 2 data. Table 7-25 indicates 
that 3, 4, 5 and 6 are good predictors of rt. Three and 4 were about the same for rms, but this table 
was left out because the TLM load estimates were not good predictors overall the algorithms. 

Table 7-26 shows that 3 accounts for all the variance when predicting subjects’ Visual Workload 
reports, with 4, 6 and 5 closely following. These results support algorithm 3 and 6 as good 
predictors for both of the Window/Panes’ experimental data sets. However, since 1 and 6 were 
good for predicting the tracking decrement scores for the Samo & Wickens data, 6 seems to be the 
best predictor over all of the tests. 
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Table 7-25: VACM load estimates regressed on Windfiw/PANES 2 rt collapsed 

across cue and no-cue conditions 



l 

2 

3 

4 

5 

6 

r-sq 

0.966 

0.789 

0.998 

0.985 

0.978 

0.979 

r 

0.983 

0.888 

0.999 

0.992 

0.989 

0.989 

EEKEBHH 

0.915 

0.648 

0.995 

0.964 

0.945 

0.948 

res ms 

229037.4 

955473.5 

11050.71 

97850.87 

148598.9 

140681.5 | 

st err est 

478.578 

977.483 

105.122 

312.811 

385.485 

375.075 

f-stat 

19.1 

5.62 

409.6 

45.6 

29.8 

31.5 

num df 

3 

2 

3 

3 

3 

3 

den df 

2 

3 

2 

2 

2 

2 


0.050 

0.096 

0.002 

0.021 

0.032 

0.030 


Table 7-26: VACM load estimates regressed on Window/PANES 2 Visual 
Workload Report collapsed across cue and no-cue conditions 



l 

5 

3 

4 

5 

6 

r-sq 

0.949 

0.751 

1 

0.997 

0.967 

0.976 

r 

0.974 

0.866 

1 

0.998 

0.983 

0.988 


0.873 

0.585 

1 

0.993 

0.917 

0.941 

res ms 

21.491 

70.510 


1.0434 

13.968 

9.9030 

st err est 

4.6359 

8.3970 


1.0215 

3.7375 

3.1469 

f-stat 

12.5 

4.53 


271.1 

19.6 

27.9 

num df 

3 

2 

3 

3 

3 

3 

den df 

2 

3 

2 

2 

2 

2 


0.074 

0.123 


0.003 

0.048 

0.034 


7.4 Final Analysis 

Based on these results of the first and second analyses and the regression analyses, the parameters 
were reduced to a final set (Figure 6-4, Table 7-27), and will be tested in a final experiment to be 
conducted in 1994. 

7.5 Experiment 5: Predicting timesharing of tracking with modality and stage 
specific decision tasks in Instrusim using TLM. 

7.5.1 Objective 

The objective of the final experiment is to test which clash pairs capture modality and dimension 
specific interactions. 

To test this, the experimental tasks need to reflect specific interactions across perceptual modalities 
and processing stages. The following design was created to fit this need. This experiment is 
currently being run using a simulation testing environment called Instrusim, developed by Dr. 
Tony Andre at Ames Research Center. 





























































































































7.5.2 Design 

Figure 7-7 depicts the design of this final experiment. The bottom two-thirds of the figure depict 
the different difficulty levels of the primary task pointing to different levels of the secondary tasks. 
The secondary tasks are broken down into their modality and dimension specific components in the 
bottom half of the figure. The top half lists the dependent measures, with the circle-arrow graphics 
delineating the data collection points. 

The design incorporates the following assumptions: 

1) Conflicts drive meaningful load values. 

Therefore, the tasks should distinguish between Conflict driven and Demand driven effects. 

2) The task classifications probably interact with the load calculation algorithms, i.e., the loads 
might be a function of the number of attributes, not the type of attributes. 

Therefore, the tracking and decision tasks should have the same or different attributes representing 
conflicts or no conflicts. If the algorithm and classifications don’t interact, then loads should 
distinguish between tasks irrespective of the number of attributes used to classify the tasks. 


7.6 Summary of Validation of All Tests 

Table 7-27: 

Final Parameters and Variations 


Taxonomy 

Conflict Values 

Attribute Pairs: 

Conflict Value Matrices: 

Full set 

8 

Matrix Values: 

Demand Values 

0-10 

Demand Value Matrices: 

Algorithms 

8 

VACM Index Selection: 

Matrix Values: 

Sequential Pairing of Tasks 

0-1 

VACM Index Integration: 

Separable 

Activity Demand & Conflict Integration: 
Product 

Matrix Combining Algorithms: 

Average 


Eleven tests were conducted using four different experimental data sets to validate the TLM, with 
one final experiment currently being planned for this winter. This last experiment will be the final 
effort on the part of the A3I project. Further validation will only take place by other researchers. 

Table 7-8 lists the factors and parameters currently incorporated into the TLM. These were selected 
based on the results of the 1 1 tests. The results were fairly consistent across all the tests, allowing 
the final set of parameters to be selected with a fair degree of confidence. 

These parameters are surprising, particularly the Algorithms. Permutations of tasks would seem 
better than Sequential Pairing because Sequential Pairing misses conflicts. Further, permutations 
would seem better than Combinations because Combinations is sensitive to order effects. The 
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driving factor seems to be that Permutations adds noise by virtue of its capturing all conflicts. The 
Conflict Value Matrices are a good addition to the TLM, because penalties are specialized on the 
type of task, which adequately represents the conflicts. Apparently though, the sheer number of 
conflicts accounted for by the Permutation of tasks still reduces the distinction between relevant 
and irrelvant tasks that conflict matrices add. 

7.6.1 Taxonomy 

Selecting the set of Taxonomic attribute pairs to classify tasks was not as clear-cut. This factor was 
only studied in test two of the first analysis: correlating TLM load estimates against those generated 
by three other workload models in a dual task setting (Samo & Wickens, 1991). The correlations 
of the TLM load estimates with the other models’ estimates and with tracking decrement scores 
indicated that the patterns of correlations across the three different sets of attribute pairs (Full, 3- 
Pair, and 2-Pair) were sufficiently similar that one could be confidently selected. There were some 
differences, but they did not appear strong enough to indicate a trend of any kind. Using 
parsimony as the only criterion would dictate selecting the 2-Pair set. However, the theory behind 
the task attribute classifiers suggests that a more detailed task representation should capture more 
task variance, resulting in better predictions. Since this factor was not studied in all tests, 
parsimony may not be the best, sole criterion in this case. On the practical side, using fewer 
attributes results in timelines that reflect very little difference in tasks across a mission scenario. 
This reduces the ability of the analyst to view task behavior and to understand what the operator is 
doing. Therefore, the Full Set of Taxonomic attributes was selected as a sufficient way to represent 
the attributes important to task performance. 

7.6.2 Demand Values 

All of the tests unequivocally indicated that binary Demand Matrix Values (0 to 1) were sufficient 
to capture the appropriate task demands. The two other value sets, 0-4 and 0-10, showed the exact 
same patterns as the binary value set. Even though the TLM was initially designed for the 0-4 set, 
parsimony dictates using binary values in the absence of other differentiating criterion. 

7.6.3 Conflict Values 

Testing the Conflict Value Factor in the first analysis showed that using global penalty values that 
were insensitive to the type of conflicting demands did not capture conflicts as well as using 
matrices of conflict values that were specialized on the type of conflict. The global values 
manipulation indicated that capturing conflicts using different values was necessary, as indicated 
by the poor correlations using a conflict value of 1 versus values of 2 or 3. The mixed results for 
values of 2 or 3 indicated that in some cases this further differentiation helped but was not 
sufficient. Therefore, the second analysis used values that were completely specified by the type of 
the conflict in order to resolve the mixed results, which they did. The correlations were higher and 
there were more significant correlations across all the tests in the second analysis (tests 5-11). 
Consequently, matrices of conflict values that ranged from 0 to 10 were selected as the best way to 
penalize conflicting demands. 

7.6.3 Algorithms 

The results of testing the Algorithm’s Factor from both the first and second analyses clearly 
indicated which parameters should be selected with the exception of the VACM Index Selection 
parameter. 

The Matrix Combining Algorithm did not differentiate between summing or averaging, so 
averaging was selected because the it was the current implementation. 
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In the first analysis, the Activity Demand and Conflict Integration Algorithm did not differentiate 
between summing or averaging the product of each demand and conflict value, but did between 
summing the product and multiplying the product, Consequently, both summing and multiplying 
were tested further in the second analysis, which indicated that the product correlated better with 
the data sets. 

The VACM Index Integration Algorithm results were clear across the tests in the two analyses, 
keep the VACM classification indices separate; In the first analysis, integrating the classifications 
before selecting the index pairs that access the appropriate demand and conflict values lost 
important distinctions between the demands and conflicts; In the second analysis, keeping the 
classifications separate while selecting the index pairs captured the relevant conflicts and demands. 


The VACM Index Selection Algorithm produced the least clear results of the Algorithm parameters 
across analyses. All of the correlations were lower in the first analysis than in the second because 
of integrating the classifications, but sequentially pairing classification indices generated load 
estimates that correlated higher than using all of the combinations of indices (permutations of 
indices were not selected in the first analysis). All of the correlations were higher in the second 
analysis than in the first because of separating the classifications. Permutations of indices were 
tested against the others, but correlated less well than either all combinations or sequential pairing. 
Regression analyses conducted in the second analysis indicated that sequential pairing of indices 
produced individual VACM load estimates that captured more variance than those generated using 
all combinations. This means that representing demands and conflicts is a trade-off. A more 
accurate and detailed representation is better up to a point, in which the sheer number of the types 
of demands and conflicts represented overshadows the more important ones. Beyond this point, 
adding details of the demands and conflicts adds error variance and reduces the correlations. 
Sequential pairing seems to best represent this trade-off, sacrificing some of the detail of the less 
important distinctions for fewer, more important ones. 

It is possible that this trade-off in detail interacts with the number of taxonomic attributes used to 
classify tasks. However, this trade-off was not tested, so currently it remains an open question. 
However, it will be tested in the final experiment sometime in early 1994. 

7.6 Conclusion 

Based on these results, the current implementation of the TLM uses 4 to 5 pairs of task attributes 
per dimension to classify tasks. These classifications determine which binary demands apply, and 
which conflict penalties between 0 and 10 apply. The sequential pairing algorithm determines 
which demands conflict and therefore which conflict penalty values will be multiplied to the 
conflicting demands. Eight load estimates (products of the demand-conflict products) are calculated 
from 16 matrices (eight demand and eight conflict). Four load estimates represent the combined 
demands and conflicts within each VACM dimension, and four represent the combined demands 
and conflicts between the Visual- Auditory, Visual-Cognitive, Auditory-Cognitive, and Cognitive- 
Motor dimensions. These eight load estimates are re-scaled using a log transformation and then 
averaged across the dimensions. The results are load estimates of the Visual, Auditory, Cognitive, 
and Motor loads that would be imposed on an operator of the design in question. 
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Figure 7-7: Design of the final TLM experiment 
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9.0 ABBREVIATIONS AND ACRONYMS 


AC bw Auditory-Cognitive between matrix. 

Acw Auditory within matrix. 

CAD Computer Aided Design. 

C cw Cognitive within matrix. 

C t ki Indices To The Rows In The Conflict Matrices 

C u ki Indices To The Columns In The Conflict Matrices 

Dfla Indices To The Rows In The Demand Matrices 

D u ici Indices To The Columns In The Demand Matrices 

Ml Equation for calculating loads using the integral, summation algorithm. 

CM(j W Cognitive-Motor between matrix. 

E a Auditory load value. 

Lc Cognitive load value. 

Ljn Motor load value. 

Ly Visual load value- 


^cw Motor load value. 

Ml Matrix load value 

MIDAS Man-Machine Design and Analysis System. 

TLM Task Load Model. 

TLAP Timeline Analysis Procedure. 

VAfow Visual-Auditory between-matrix. 

VACM Visual Auditory Cognitive Motor. 

VACP Visual Auditory Cognitive Psycho-Motor. 

VCb w Visual-Cognitive between-matrix- 

^cw Visual within-matrix.- 

W/INDEX Workload Index 

Window/P ANES Workload/PerformANcE Simulation 
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10.0 GLOSSARY 


Additional-Cost 

An unknown function or constant that accounts for the obtained increases in the magnitude 
of workload predicted by an averaging model. 

Separable Permutation Algorithm 

An algorithm that sums the pairwise products of all possible combinations of elements 
within each activity and then sums these products across all concurrent activities 

Attention 

A mental control mechanism that guides, focuses, or elaborates the acquisition and 
processing of information. 

Auditory 

A modality of perception involving aural stimuli. 

Auditory -Cognitive 

The interaction of aural perceiving and central processing mechanisms. 

Auditory-Visual 

The interaction of aural perceiving and visual perceiving. 

Averaging Model 

A model that predicts retrospective workload ratings by averaging the difficulty of the 
events or stimuli that are experienced. 

Behavioral State 

A description of the attributes and values that constitute a specific pattern of human 
performance. 

Best-Fitting Model 

A model that best describes the obtained patterns of human performance or behavioral 
states. 

Between-Matrix 

A two dimensional matrix of values that represents the demands incurred on the human 
information processing system from the interaction of information processing mechanisms 
in different stages. 

Cognitive 

The central processing stage composed of at least the attentional, transformational, and 
memory mechanisms and processes that act on perceived information and which prepare 
the information processing system to generate a response to stimuli if necessary. 

Cognitive-Motor 

The interaction of central processing mechanisms and motor response mechanisms. 
Conflict Matrix 

A two dimensional matrix of values that represents the demands and conflicts among the 
structural and procedural psychological attributes that describe an activity. 

Conflict Value 

A specific value that represents the interaction between two structural or procedural 
psychological attributes classifying an activity. 

Dimension 

A top level set of structural or procedural psychological attributes that corresponds to a 
specific stage of information processing. This is the level at which load-values are assigned 
to an activity. 
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A bottom level structural or procedural psychological attribute that is used to classify an 
activity. 

Event Based . , , , 

A simulation that progresses according to changes m state variables that represent specitic 

events that occur in the environment or behavior of the operator. 


Genera 8.1 

The Symbolics Lisp operating system. 

Human Information Processing . 

The human mental activities and structures that represent and manipulate information 
symbolically, and which enable humans to perceive and respond to changes in external 
(environmental) or internal (mental) states. 

Human Performance Model , . . 

A quantitative (analytic or computer-based) representation or description or all or parts ot 

human operators or maintainers of complex, dynamic systems. 

Invariant Property f 

A characteristic of the information processing system that does not change as a function ot 

the information extracted from the perceptual array. 


Load Balancing Strategy . 

A behavioral change in an operator as a function of the imposed visual, auditory, cognitive 
and motor loads imposed by performing an activity. The effect of the behavioral change is a 
regression to the mean workload (reduction of peak loads) of the pertinent activities with 
regard to allotted time. 

Load Value , . 

A value that represents the amount of psychological resources required to perform an 
activity relative to performing another activity or set of activities. 


Memory 

The psychological mechanisms that maintain information over time. 


Mental Workload ... r , 

An evaluation about the difficulty of ongoing experiences and the impact of those 
experiences on the physical and mental states of an operator. The evaluation is a function of 
the collection of attributes, that may or may not be relevant, controlling the evaluations or 
behavior that depend on the circumstances and design of a given activity(s) and the a priori 
bias of the operator. 


Motor 

The effector mechanisms of the human body. 


Multiplicative Algorithm . . , . . 

An algorithm that sums all the possible combinations of pairwise products of elements 
within and between all concurrent activities. 


Object-Oriented Programming 

Programming languages that represent data structures as objects with attributes and values. 

Normative Model „ ... 

A model that predicts how an operator should perform by assuming rational operator 

behavior. 


Phase IV 

The period of research and development on MIDAS from the small offsite in March 1989 to 
the end of demos in July 1990. 
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Phase V 

The period of research and development on MIDAS from the large offsite in November 
1990 to the end of demos in June 1992. 

Physical System 

The design of the system hardware and software. 

Process 

A specific information processing mechanism that manipulates information symbolically. 
Resource 

The attentional, physical and memory capabilities of an operator. 

Scheduling Model 

The software configuration item that sequences the order of the simulated activities. 
Serial Constraints 

The imposed order of interactions between the dimensions used in the TLM. 

Simulation Executive 

The software configuration item used to control the simulation by controlling the flow of 
execution of the rest of MIDAS' software configuration items. 

Software Component 

A component of a software architectural unit/item used to implement a specific model or 
tool within MIDAS. 


Structure 

A specific information processing mechanism that represents information symbolically. 


Summing Model 

A model that predicts retrospective workload ratings by summing the difficulty of the 
events or stimuli that are experienced. 


Activity Decomposition Model 

The software configuration item that decomposes high level goals into the simulated 
activities. 


Task Loading Model 

The software configuration item that predicts the visual, auditory, cognitive, and motor 
loads that the simulated activities impose on the operator of the system. 

Tick Based 

A simulation that progresses according to changes in state variables that represent specific 
events that occur in the environment or behavior of the operator. 


User Interface 

The commands and displays that an analyst or user of MIDAS uses to interact with 


MIDAS. 


Variant Property 

A characteristic of the information processing system that changes as a function of the 
information extracted from the perceptual array. 

Visual 

A modality of perception involving visual stimuli. 

Visual-Cognitive 

The interaction of visual perceiving and central processing mechanisms. 

Within-Matrix 

A two dimensional matrix of values that represents the demands incurred on the human 
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information processing system from the interaction of information processing mechanisms 
within the same stage. 
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